Unverified50% confidenceFactExact time
Experiments were conducted on MMLU-Pro and MMMU-Pro, and the results show that poorly calibrated self-confidence can still support statistically valid stratified auditing.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/27/2026
First Seen
Valid until: 1/25/2027
Sources
Related Entities
Related Claims
Unverified模型在标准化评测基准（如HumanEval、MMLU）上的高分表现与真实工作场景中的可靠性之间存在显著差距76% similarUnverifiedMultiple academic studies have shown that models scoring highest on mainstream benchmarks such as MMLU and HumanEval often perform significantly worse on novel questions of equivalent difficulty.72% similarUnverifiedThe 2023 TruthfulQA benchmark showed that even the strongest models at the time did not exceed 60% accuracy on questions requiring precise factual judgment.69% similarVerifiedOn mainstream evaluations such as MMLU, HumanEval, and GSM8K, the score gaps among leading large models are often only a few percentage points.68% similarVerifiedA model with slightly lower accuracy but well-calibrated confidence is often more suitable for production deployment than a model with higher accuracy that is blindly overconfident.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1103044API
curl https://kongchang.com/api/v1/knowledge/claims/1103044MCP
get_claim(id=1103044)