End-to-End Latency
A core performance metric for voice agents, referring to the time between when the user finishes speaking their last word and when the agent begins playing its first segment of speech. It consists of four cascaded components: STT, the LLM's first token, the TTS first packet, and network overhead.
Core Facts
Timeline (last 90 days)
End-to-end latency is typically the sum of capture, network transmission, ASR, LLM time to first token, and TTS time to first packet.
Time to first byte (TTFB) refers to the time from sending text to receiving the first audio byte; end-to-end latency includes the entire time consumed by audio capture, ASR transcription, intent processing, TTS synthesis and playback buffering.
When a single tool call itself is the bottleneck (max(tᵢ) is large), continuing to increase concurrency yields almost no improvement in total latency and only increases resource pressure.
When executed concurrently (concurrency ≥ N), the total time for N independent tool calls is approximately max(tᵢ).
Assume that within one round there are N mutually independent tool calls, each taking time t₁…t_N, and the total time for serial execution is approximately Σtᵢ.
Appointment conversation scenarios expect the end-to-end latency from when the user finishes speaking to when the agent begins responding to be controlled within hundreds of milliseconds.
Time to First Audio (TTFA) refers to the time elapsed from text input to the output of the first frame of playable audio, and is a core metric for measuring the response speed of streaming TTS systems.
All Facts (7)
Time to First Audio (TTFA) refers to the time elapsed from text input to the output of the first frame of playable audio, and is a core metric for measuring the response speed of streaming TTS systems.
75%UnverifiedEnd-to-end latency is typically the sum of capture, network transmission, ASR, LLM time to first token, and TTS time to first packet.
50%UnverifiedTime to first byte (TTFB) refers to the time from sending text to receiving the first audio byte; end-to-end latency includes the entire time consumed by audio capture, ASR transcription, intent processing, TTS synthesis and playback buffering.
50%UnverifiedWhen executed concurrently (concurrency ≥ N), the total time for N independent tool calls is approximately max(tᵢ).
50%UnverifiedAssume that within one round there are N mutually independent tool calls, each taking time t₁…t_N, and the total time for serial execution is approximately Σtᵢ.
50%UnverifiedWhen a single tool call itself is the bottleneck (max(tᵢ) is large), continuing to increase concurrency yields almost no improvement in total latency and only increases resource pressure.
50%UnverifiedAppointment conversation scenarios expect the end-to-end latency from when the user finishes speaking to when the agent begins responding to be controlled within hundreds of milliseconds.
50%