Verified80% confidenceFactTime unknown
Whisper's architecture adopts an encoder-decoder structure, where the encoder transforms the mel spectrogram of audio into high-dimensional feature representations, and the decoder generates text output word by word in an autoregressive manner.
6
Sources
80%
Confidence
Long-term
Relevance
5/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe text encoder is responsible for converting natural language prompts into high-dimensional embedding vectors for use by the diffusion model's denoising process82% similarUnverifiedWhisper unifies speech recognition, language identification, timestamp prediction, and speech translation into a sequence-to-sequence task, specifying the task type via special control tokens.75% similarUnverifiedIf the input is audio, the Vision Encoder can be replaced with a Speech Encoder to encode and align the data before passing it to the large language model.74% similarVerifiedNeural audio codecs such as EnCodec and DAC compress continuous audio into discrete token sequences, enabling speech to be directly processed by autoregressive language models just like text tokens.74% similarUnverifiedThe architecture of FD-VAD consists of a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/8513API
curl https://kongchang.com/api/v1/knowledge/claims/8513MCP
get_claim(id=8513)