Unverified50% confidenceOpinionExact time
There are often differences in criteria between self-built evaluation sets and public benchmarks, and factors such as the difficulty distribution of the test set and the number of answer candidates can significantly affect absolute scores, so 87.2 is more suitable as an internal acceptance signal for the team rather than a basis for cross-comparison.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/27/2026
First Seen
Valid until: 1/25/2027
Sources
Related Entities
Related Claims
UnverifiedThe Arena user-voting-based evaluation method is closer to real-world usage scenarios compared to traditional static benchmarks, and is also harder to game through targeted optimization.78% similarVerifiedThere is a gap between benchmark scores and real user experience, and some vendors engage in targeted optimization for specific benchmarks (score gaming) as well as training data contamination that inflates scores.78% similarUnverifiedSensitivity stemming from the evaluation method may arise from the scoring rubric favoring certain phrasings, a single reference answer format, or an automatic evaluator that parses output format too rigidly.75% similarUnverifiedRelative scoring is a horizontal comparison of individuals within the same organization (such as percentile ranking); its scores drift as team composition changes and do not directly correspond to external standards.74% similarUnverifiedWhen a review combines a low score with high confidence and the criticism is unjustified, it amplifies the gap between the review opinion and the actual evidence, making it easier for the AC to identify it as a review quality issue.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1102932API
curl https://kongchang.com/api/v1/knowledge/claims/1102932MCP
get_claim(id=1102932)