STT

A complete voice AI interaction involves three serial processing stages: STT, LLM inference, and TTS.

传统语音AI普遍采用级联式架构，先用ASR将语音转文字，再交给语言模型处理，最后用TTS合成输出

ODS integrates voice interaction capabilities, supporting speech-to-text (STT) and text-to-speech (TTS)

Early ChatGPT voice used a cascaded architecture: first converting speech to text, then having the large language model generate a response, and finally converting it to speech for output.

End-to-end latency in voice AI typically involves three stages: speech-to-text (STT) recognition, large language model inference, and text-to-speech (TTS) synthesis

When selecting components, the metrics Reimer focuses on include: the audio framework's input and output latency, STT's confidence time, the LLM's time to first sentence (a combination of time to first token and tokens per second), and TTS's time to first audio (which must be measured up to the point where the user actually hears sound).

AstroBeam's parallelization approach is: without waiting for STT to fully confirm, start the LLM early as soon as there is some confidence that the user has probably finished speaking and that the transcription is roughly correct; on the LLM side, send the first speakable chunk to TTS immediately; and have TTS stream it out to the user.

When evaluating Arabic STT, one cannot look only at word error rate (WER) but must also look at key entity recognition rates (station names, ticket numbers, dates, amounts, etc.), because entity recognition errors will crash all downstream processes.

Narrowband telephony audio (8kHz) loses high-frequency information, further lowering Arabic speech recognition accuracy.

Traditional remote translation solutions typically use a three-stage serial pipeline of speech recognition (STT), LLM translation, and speech synthesis (TTS), which suffers from compounded latency, loss of intonation, and inconsistent voice timbre.

In the above measured data, the data and judgment times reflect only text judgment results and do not include the overhead of speech recognition (STT) and speech synthesis (TTS).

Low latency and high recognition accuracy involve the selection of Voice Activity Detection (VAD) and Speech-to-Text (STT) engines

STT models in low signal-to-noise ratio scenarios do not admit that they cannot hear clearly; instead, they tend to fabricate plausible but incorrect words and phrases (producing hallucinations)

Adding preprocessing steps such as noise reduction, dereverberation, volume normalization, and VAD before audio enters the STT system can significantly improve downstream transcription quality.

Different TTS engines each have their own strengths and weaknesses in naturalness, latency, and language coverage, while different STT engines vary in performance on noisy environments and accent recognition

When building voice products, developers typically need to stitch together STT, LLM, and TTS services from multiple vendors, and integrating each provider means a separate set of API documentation, authentication, billing, and error-handling logic

An AI agent capable of making phone calls typically consists of three layers: speech recognition (STT), a conversational brain (LLM), and speech synthesis (TTS).

The core idea of the sandwich architecture is to place a large language reasoning model (LLM) in the middle, with an STT model added in front for speech recognition and a TTS model added at the back for speech output.

This project achieved a response latency of approximately 0.3 seconds through fine-grained optimization of the front-end STT and back-end TTS.

语音智能体技术栈通常包含STT、LLM推理、TTS三个串行环节，STT约耗时200-400ms，LLM首token延迟约100-500ms，TTS生成约100-300ms