Unverified50% confidenceFactExact time
Some vendors tend to over-optimize for evaluation sets (benchmark gaming/benchmark contamination), leading to inflated scores but insufficient generalization ability.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedSome models carry the risk of over-optimization toward test sets, causing leaderboard scores to deviate from real-world performance (benchmark saturation).73% similarUnverifiedVendor optimization targeting specific benchmarks causes benchmark scores to rise while becoming decoupled from improvements in real-world capability, a problem known as benchmark hacking.72% similarVerified若用含测试集的全量数据计算均值和标准差，会导致预处理泄漏，使评估指标虚高，无法真实反映模型泛化能力72% similarUnverified评分校准是关键技术难点，模型必须能大量给出低分，否则过滤系统失去意义71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/829389API
curl https://kongchang.com/api/v1/knowledge/claims/829389MCP
get_claim(id=829389)