Verified90% confidenceTradeoffExact time
There is a gap between benchmark scores and real user experience, and some vendors engage in targeted optimization for specific benchmarks (score gaming) as well as training data contamination that inflates scores.
8
Sources
90%
Confidence
Long-term
Relevance
7/13/2026
First Seen
Sources
Gemini 3.1 Pro体验：共情式对话为何让用户离不开它
redditr/Bard7/10/2026
Related Claims
UnverifiedHigh scores on benchmarks do not necessarily equate to faithful execution of user requirements, and the practice of using benchmark scores as the sole quality signal carries fundamental risks.79% similarUnverifiedThere are often differences in criteria between self-built evaluation sets and public benchmarks, and factors such as the difficulty distribution of the test set and the number of answer candidates can significantly affect absolute scores, so 87.2 is more suitable as an internal acceptance signal for the team rather than a basis for cross-comparison.78% similarUnverifiedThere exists an evaluation-application gap between benchmark performance and actual production environment performance, and some vendors tend to over-optimize for evaluation sets (benchmark gaming / benchmark contamination).76% similarUnverifiedPublicly disclosing post-training metrics involves a trade-off: transparency helps build technical credibility, but training recipes and data composition are core competitive barriers for vendors, and excessive disclosure may erode one's own advantage.76% similarUnverifiedBehavioral label data has inherent defects, conflating candidate exposure opportunities, recruiter subjective preferences, and objective qualifications, making it difficult to isolate the true influence of each factor75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/501618API
curl https://kongchang.com/api/v1/knowledge/claims/501618MCP
get_claim(id=501618)