MMLU Pro
MMLU Pro（Massive Multitask Language Understanding Pro）是一个大规模多任务语言理解基准测试，是原版MMLU的增强版本。该基准涵盖数学、科学、工程、法律、人文等多个学科领域，通过更具挑战性的多项选择题评估大型语言模型的综合知识理解与推理能力。相较于原版MMLU，其题目难度更高、选项更多，旨在更精准地区分顶尖语言模型的性能差异。
Timeline (last 90 days)
The 2B-parameter Gemini Nano model is nearly on par with last year's 27B model on the MMLU Pro benchmark.
In the MMLU Pro test, 8-bit narrowly beat OQ4E with 60.6% versus 59.9%.
The NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.
All Facts (4)
MMLU Pro涵盖57个学科领域，人类专家平均得分约为78分
75%UnverifiedThe 2B-parameter Gemini Nano model is nearly on par with last year's 27B model on the MMLU Pro benchmark.
50%UnverifiedIn the MMLU Pro test, 8-bit narrowly beat OQ4E with 60.6% versus 59.9%.
50%UnverifiedThe NVFP4 version of Qwen3-27B shows only marginal loss compared to uncompressed FP16 on the MMLU Pro and GPQA Diamond benchmarks.
50%