MMLU-Pro
An enhanced version of the original MMLU benchmark that expands the number of answer choices from four to ten and incorporates more questions requiring deep reasoning rather than recall. These modifications make it substantially harder to achieve high scores through guessing alone.
Timeline (last 90 days)
Experiments were conducted on MMLU-Pro and MMMU-Pro, and the results show that poorly calibrated self-confidence can still support statistically valid stratified auditing.
MMLU-Pro was introduced by TIGER-Lab in 2024, changing the four-choice format to ten-choice, with about 12,000 reasoning-intensive questions in total, and is one of the knowledge benchmarks commonly cited for current open-source models.
The top five in MMLU-Pro are separated by only 0.6 percentage points: Darwin-180B-RSI 88.1%, MiniMax-M2.1 88.0%, Intern-S2-Preview 88.0%, Qwen3.5-397B-A17B 87.8%, DeepSeek-V4-Pro 87.5%.
As of October 19, 2026, Darwin-180B-RSI ranks first on the AIhoster leaderboard for MMLU-Pro at 88.1% and first for GPQA Diamond at 94.4%.
On reasoning tasks such as MMLU-Pro and TriviaQA, U-Space's single-forward-pass scorer consistently ranks correct answers ahead of entropy methods or ensemble methods.
The top of MMLU-Pro is FINAL-Bench/Darwin-180B-RSI with a score of 88.12; it also takes first place on gpqa with a score of 94.44.
GPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
Version 2.0 shows declines in knowledge-based capabilities (MMLU-Pro 72.5 vs. base 78.5) and competition-level coding ability (LiveCodeBench 51 vs. 56).
On the MMLU-Pro benchmark, Flash-Next IQ4_XS scored 83.8% and Qwen3 27B FP8 scored 75.0%.
2 more timeline events
All Facts (11)
Experiments were conducted on MMLU-Pro and MMMU-Pro, and the results show that poorly calibrated self-confidence can still support statistically valid stratified auditing.
50%UnverifiedMMLU-Pro was introduced by TIGER-Lab in 2024, changing the four-choice format to ten-choice, with about 12,000 reasoning-intensive questions in total, and is one of the knowledge benchmarks commonly cited for current open-source models.
50%UnverifiedThe top five in MMLU-Pro are separated by only 0.6 percentage points: Darwin-180B-RSI 88.1%, MiniMax-M2.1 88.0%, Intern-S2-Preview 88.0%, Qwen3.5-397B-A17B 87.8%, DeepSeek-V4-Pro 87.5%.
50%UnverifiedAs of October 19, 2026, Darwin-180B-RSI ranks first on the AIhoster leaderboard for MMLU-Pro at 88.1% and first for GPQA Diamond at 94.4%.
50%UnverifiedOn reasoning tasks such as MMLU-Pro and TriviaQA, U-Space's single-forward-pass scorer consistently ranks correct answers ahead of entropy methods or ensemble methods.
50%UnverifiedThe top of MMLU-Pro is FINAL-Bench/Darwin-180B-RSI with a score of 88.12; it also takes first place on gpqa with a score of 94.44.
50%UnverifiedGPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
50%Unverified在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
50%UnverifiedVersion 2.0 shows declines in knowledge-based capabilities (MMLU-Pro 72.5 vs. base 78.5) and competition-level coding ability (LiveCodeBench 51 vs. 56).
50%UnverifiedThe evaluation covered five benchmarks: MMLU-Pro, GPQA Diamond, GSM8K, German MGSM, and IFEval.
50%UnverifiedThe score gap among multiple frontier models on MMLU has narrowed to the order of one percentage point, driving the emergence of more challenging benchmarks such as MMLU-Pro and ARC-AGI.
50%