Whisper
Whisper is an automatic speech recognition (ASR) model developed and open-sourced by OpenAI that transcribes audio content into text and supports speech recognition and translation across multiple languages. Trained on large-scale multilingual speech data, the model exhibits strong noise robustness and can handle a variety of accents and speaking styles. It is applicable to use cases such as subtitle generation, voice assistants, and meeting transcription.
Core Facts
Whisper is an automatic speech recognition model open-sourced by OpenAI in 2022, built on a Transformer architecture and trained on 680,000 hours of multilingual audio data.
Azure OpenAI Service provides enterprise customers with the ability to invoke models such as GPT-4, DALL-E, and Whisper in a compliant environment.
The core architecture of multimodal LLMs typically consists of three components: a visual encoder (such as CLIP or ViT), a cross-modal alignment layer, and a language decoder
Timeline (last 90 days)
In the UP host's test, Whisper's Base model output Traditional Chinese with severely degraded quality (garbled text at the end), while the Small model remained coherent up to the 10th minute but had homophone errors (such as hearing Word2Vec as WordOfTough).
GRPO's advantage over SFT holds consistently across four Whisper model sizes, four teacher model families, three domains, and four languages.
When using pseudo-labels generated by Whisper in the banking domain, GRPO achieved relative WER improvements of 29.3% to 56.2% compared with SFT.
Local deployment of Whisper is completely free, but it requires a GPU or a relatively powerful CPU, and inference speed depends on the hardware.
Mistral claims that Voxtral 3 leads Whisper and ElevenLabs in accuracy, but this conclusion comes only from the company's own charts and has no independent benchmark support yet.
Whisper has long been regarded as the baseline solution in the speech transcription field, and the community has formed a default reference of "transcription quality good enough, cost manageable."
Parakeet does not support translation or Hindi, so sites that need these features still need to keep Whisper.
Maxwell has been used on computationally intensive AI models such as Whisper (speech recognition) and LTX-2 (video generation).
Whisper can run fully offline: once the model weights are downloaded, inference requires no internet connection.
walkai-talkai uses the Whisper model for local speech transcription on Mac, and only the transcribed text commands are sent to cloud models (if a cloud API is selected).
40 more timeline events
All Facts (20)
Whisper is an automatic speech recognition model open-sourced by OpenAI in 2022, built on a Transformer architecture and trained on 680,000 hours of multilingual audio data.
90%VerifiedAzure OpenAI Service provides enterprise customers with the ability to invoke models such as GPT-4, DALL-E, and Whisper in a compliant environment.
85%VerifiedThe core architecture of multimodal LLMs typically consists of three components: a visual encoder (such as CLIP or ViT), a cross-modal alignment layer, and a language decoder
80%VerifiedThe core design of the OpenAI API uses /v1/chat/completions as the main endpoint.
80%VerifiedWhisper's architecture adopts an encoder-decoder structure, where the encoder transforms the mel spectrogram of audio into high-dimensional feature representations, and the decoder generates text output word by word in an autoregressive manner.
80%VerifiedWhisper is available in multiple sizes ranging from tiny (39M parameters) to large-v3 (1.5B parameters).
80%VerifiedA voice agent technology stack typically consists of three serial components: STT (such as Whisper), LLM inference, and TTS (such as ElevenLabs).
75%VerifiedWhisperFlow is a free tool built on OpenAI's Whisper speech recognition model.
70%VerifiedApple provides the Speech Framework to support on-device speech recognition, but its capabilities are mainly limited to transcription and lack an LLM polishing step.
65%VerifiedThe Whisper model is available in multiple sizes ranging from tiny to large and can run on local devices.
65%VerifiedOpenAI's open-source Whisper model comes in multiple size variants, ranging from the tiny version with 39M parameters to the large-v3 version with 1,550M parameters, supporting nearly 100 languages.
65%VerifiedAlthough Whisper supports multilingual recognition, its performance on intra-sentential code-switching remains far below that in monolingual scenarios.
65%VerifiedThe Whisper community has spawned variants such as whisper.cpp (a C/C++ port that runs efficiently on CPU) and faster-whisper (an accelerated version based on CTranslate2)
65%VerifiedModels such as Whisper and Wav2Vec 2.0 learn general acoustic representations on large amounts of unlabeled audio through self-supervised pre-training, reducing reliance on manually annotated data.
65%VerifiedResearch shows that Whisper's probability of hallucination is particularly prominent during silent segments, producing fluent transcription results whose content does not match the original audio.
65%VerifiedOpenAI's open-source Whisper model achieves near-human-level word error rates across 99 languages and natively supports timestamp output.
65%VerifiedQuantized versions of Whisper (such as whisper.cpp and faster-whisper) can run speech recognition in real time on consumer-grade CPUs.
65%VerifiedText models supported by the Transformers framework include GPT, BERT, LLaMA, and T5; vision models include ViT, CLIP, and Stable Diffusion; audio models include Whisper and Wav2Vec; and multimodal models include LLaVA and Qwen-VL.
65%VerifiedWhisper offers six language models: tiny (39 million parameters), base (74 million), small (244 million), medium (769 million), large (1.55 billion), and turbo (809 million).
65%VerifiedWhisper基于Transformer编码器-解码器架构，在68万小时的多语言标注音频数据上进行弱监督训练
65%