Big Bench Audio
A benchmark designed to evaluate AI models' audio reasoning capabilities, measuring overall model performance across audio comprehension and reasoning tasks.
Timeline (last 90 days)
ElevenLabs launched the real-time speech-to-speech models Max and Mimi; Max scored 98.8% on Big Bench Audio and begins speaking in about 0.57 seconds.
The Extended Thinking version scored 68.6% on the τ-Voice benchmark, 35.1% on Sierra's τ-Voice-banking benchmark, and 97.7% on Big Bench Audio.
According to official data, ThunderPhone's Storm configuration achieved a score of 99.4% on the Big Bench Audio benchmark.
The Big Bench Audio benchmark has been publicly released on Hugging Face and is available for third parties to download and independently reproduce tests.
The Big Bench Audio benchmark used by ThunderPhone has been publicly released on Hugging Face, allowing third parties to independently reproduce test results.
There may be a gap between academic benchmarks (such as Big Bench Audio) and real-world call performance in noisy environments, and the actual deployment effectiveness of AI voice products requires more user case validation.
ThunderPhone's Storm advanced tier configuration achieved a score of 99.4% on the Big Bench Audio benchmark.
All Facts (6)
According to official data, ThunderPhone's Storm configuration achieved a score of 99.4% on the Big Bench Audio benchmark.
50%UnverifiedThe Big Bench Audio benchmark has been publicly released on Hugging Face and is available for third parties to download and independently reproduce tests.
50%UnverifiedThunderPhone's Storm advanced tier configuration achieved a score of 99.4% on the Big Bench Audio benchmark.
50%UnverifiedThe Big Bench Audio benchmark used by ThunderPhone has been publicly released on Hugging Face, allowing third parties to independently reproduce test results.
50%UnverifiedThere may be a gap between academic benchmarks (such as Big Bench Audio) and real-world call performance in noisy environments, and the actual deployment effectiveness of AI voice products requires more user case validation.
50%UnverifiedElevenLabs launched the real-time speech-to-speech models Max and Mimi; Max scored 98.8% on Big Bench Audio and begins speaking in about 0.57 seconds.
50%