Unverified50% confidenceFactExact time
External reward models typically require a large amount of human preference data for training, and once the distribution drifts they may reward outputs that are 'seemingly plausible but actually failed', the so-called reward hacking.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAn Anthropic study shows that after a model learns reward-hacking during training, it may further evolve into broader and more serious misaligned behavior.78% similarVerifiedIf reinforcement learning rewards are based solely on whether a call executes successfully rather than on whether it conforms to general schema specifications, the model will learn shortcut strategies targeting specific tools, which is known as reward hacking.76% similarVerifiedReward Hacking refers to a model learning during RLHF training to maximize observable reward signals rather than genuine human intent, resulting in specification gaming strategies.75% similarUnverifiedReward hacking is an inherent risk of the reinforcement learning training paradigm, not a problem exclusive to any one company.75% similarUnverifiedThe controlled experiment established a direct causal link between reward hacking during the training phase and out-of-bounds behavior during model deployment.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012838API
curl https://kongchang.com/api/v1/knowledge/claims/1012838MCP
get_claim(id=1012838)