Time to First Token
An inference performance metric for large models that measures the time from when a user issues a request to when the model outputs its first token, reflecting the efficiency of the prefill stage.
Core Facts
Time to First Audio (TTFA) refers to the time elapsed from text input to the output of the first frame of playable audio, and is a core metric for measuring the response speed of streaming TTS systems.
The floating-point operation count of prefill grows approximately quadratically with input length, and is therefore the main source of TTFT (time to first token).
Timeline (last 90 days)
Dictation requires streaming recognition: audio is continuously fed into the model in windows of tens of milliseconds, with recognition and output happening simultaneously, placing extremely high demands on first-character latency and sentence-segmentation stability.
The floating-point operation count of prefill grows approximately quadratically with input length, and is therefore the main source of TTFT (time to first token).
Two common metrics for evaluating local model speed are time to first token (TTFT) and generation speed (tokens/s).
Time to First Audio (TTFA) refers to the time elapsed from text input to the output of the first frame of playable audio, and is a core metric for measuring the response speed of streaming TTS systems.
All Facts (4)
Time to First Audio (TTFA) refers to the time elapsed from text input to the output of the first frame of playable audio, and is a core metric for measuring the response speed of streaming TTS systems.
75%VerifiedThe floating-point operation count of prefill grows approximately quadratically with input length, and is therefore the main source of TTFT (time to first token).
65%UnverifiedDictation requires streaming recognition: audio is continuously fed into the model in windows of tens of milliseconds, with recognition and output happening simultaneously, placing extremely high demands on first-character latency and sentence-segmentation stability.
50%UnverifiedTwo common metrics for evaluating local model speed are time to first token (TTFT) and generation speed (tokens/s).
50%