MMMU-Pro
An enhanced version of the MMMU (Multimodal Understanding) benchmark, focused on evaluating large multimodal models on expert-level tasks that require joint visual and textual understanding.
Timeline (last 90 days)
Experiments were conducted on MMLU-Pro and MMMU-Pro, and the results show that poorly calibrated self-confidence can still support statistically valid stratified auditing.
GPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
All Facts (2)
Experiments were conducted on MMLU-Pro and MMMU-Pro, and the results show that poorly calibrated self-confidence can still support statistically valid stratified auditing.
50%UnverifiedGPQA Diamond, MMLU-Pro, Humanity's Last Exam, SWE-bench Verified, Terminal-Bench, MMMU-Pro, LEXam, and Open ASR Leaderboard are all official benchmarks marked by Hugging Face as official benchmark.
50%