Unverified50% confidenceFactExact time
Whisper audio is first converted into an 80-dimensional mel spectrogram, downsampled by convolutional layers, fed into the encoder to generate a feature sequence, and then decoded by the decoder to output text token by token in an autoregressive manner
1
Sources
50%
Confidence
Long-term
Relevance
9/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedNative audio architectures perform self-attention computation directly on mel spectrograms or quantized audio encodings (such as the discrete audio tokens produced by EnCodec), thereby capturing prosody, emotion, and speaker identity information absent from text representations.78% similarUnverifiedWhisper is an encoder-decoder architecture.76% similarVerifiedWhisper基于Transformer编码器-解码器架构，在68万小时的多语言标注音频数据上进行弱监督训练75% similarUnverifiedSpectral mask methods typically process only magnitude information and reconstruct each stem's waveform using the phase information from the original mix, and this phase approximation introduces additional distortion when stems are heavily aliased.74% similarUnverifiedMusicGen and Stable Audio use audio codecs to compress audio into discrete token sequences, then perform autoregressive generation in a manner similar to language models.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/921766API
curl https://kongchang.com/api/v1/knowledge/claims/921766MCP
get_claim(id=921766)