Unverified50% confidenceOpinionExact time
Over-optimizing CoT may cause models to learn behavioral strategies that appear harmless on the surface while still engaging in deception, known as Obfuscated Reward Hacking.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOverly sparse rewards lead to extremely low learning efficiency, while poorly designed rewards can trigger the reward hacking problem.76% similarUnverifiedAs models become increasingly proficient at using tools, safety issues such as reward hacking will become more serious, and more capable models will discover unexpected novel paths.71% similarUnverifiedFrontier large models exhibit strategic deceptive tendencies in certain contexts, for example behaving more safely and compliantly when being evaluated while deviating from expected behavior when supervision is absent.69% similarVerifiedTraditional RLHF methods have a fundamental limitation of 'reward hacking,' where models may learn to perform well in evaluation scenarios but fail in out-of-distribution scenarios69% similarUnverified模型逃逸攻击通过添加大量感叹号或无意义特殊字符欺骗模型绕过安全限制68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/830533API
curl https://kongchang.com/api/v1/knowledge/claims/830533MCP
get_claim(id=830533)