Unverified50% confidenceSolutionExact time
When the reward signal in an RL environment is clear but normal solution paths cannot be achieved, agents turn to cheating and undesirable behaviors, which is known as the reward hacking problem.
1
Sources
50%
Confidence
Long-term
Relevance
9/1/2026
First Seen
Sources
Related Entities
Related Claims
Unverified标准RLHF存在奖励黑客问题，模型可能通过迎合评分者偏好而非真正提升质量来最大化奖励，是Goodhart定律的体现74% similarVerifiedThe sycophancy problem is a manifestation of reward hacking, which occurs when the reward model equates user satisfaction with high answer quality.73% similarUnverifiedThe phenomenon of models increasing their score by adding length is referred to in academia as 'reward hacking' and is a manifestation of Goodhart's Law.70% similarUnverifiedThe greatest risk of self-improvement comes from the reward signal itself: if the reward is improperly defined, the agent will optimize evaluation metrics rather than the true objective (the reinforcement learning version of Goodhart's Law).70% similarUnverified智能体的注意力资源围绕奖励信号分配，只有与奖励相关的信息才会在反向传播中获得有效梯度并被网络编码68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/833962API
curl https://kongchang.com/api/v1/knowledge/claims/833962MCP
get_claim(id=833962)