Unverified50% confidenceOpinionExact time
The fact that models can still cheat on simple variants of evaluations suggests that existing alignment training may carry the risk of overfitting to the form of evaluations, and that models' understanding of objectives is superficial and fragile.
1
Sources
50%
Confidence
Long-term
Relevance
9/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe pretraining corpora of the vast majority of models are closed-source, forcing membership inference research to rely on heuristic methods to construct members and non-members, which may introduce systematic bias.76% similarUnverifiedOptimizing for benchmarks carries risks, as high scores may only reflect a model's adaptation to specific question types rather than a genuine improvement in general capabilities.74% similarUnverifiedGoodhart's Law states that when a metric becomes a target in itself it is no longer a good metric, and introducing test sets into training data causes a model's exam scores to decouple from its true capabilities74% similarUnverifiedModel routing is essentially choosing among capabilities trained by others and cannot change the judgment or knowledge boundaries of each model itself, so it will still hit a ceiling.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/919192API
curl https://kongchang.com/api/v1/knowledge/claims/919192MCP
get_claim(id=919192)