Unverified50% confidenceFactExact time
If a benchmark rewards "giving an answer" without penalizing "wrong answers," guessing is statistically a worthwhile strategy.
1
Sources
50%
Confidence
Long-term
Relevance
9/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe greatest risk of self-improvement comes from the reward signal itself: if the reward is improperly defined, the agent will optimize evaluation metrics rather than the true objective (the reinforcement learning version of Goodhart's Law).72% similarUnverifiedHuman annotators exhibit a cognitive bias toward more information being better when evaluating answers, causing reward models to implicitly link length with quality, which is one of the causes of model verbosity.72% similarUnverifiedIncentive misalignment is referred to in academic economics as "measurability bias," meaning that evaluation systems tend to reward easily quantifiable outputs.71% similarUnverifiedThe confidence threshold mechanism is closely related to the academic concept of Calibration; a good model not only needs to answer correctly, but also needs to know when it might answer incorrectly.71% similarUnverifiedThe practical value of such evaluation optimization tools is highly dependent on the quality of the evaluation dataset and the reasonableness of the business metric definitions; if the input data fails to represent real scenarios or the metric design is biased, misleading conclusions may result.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/955809API
curl https://kongchang.com/api/v1/knowledge/claims/955809MCP
get_claim(id=955809)