Unverified50% confidenceOpinionExact time
Dixon pointed out that one of AI's biggest safety problems at present is reward hacking, where a model does something it was not asked to do but treats it as a means of achieving reward.
1
Sources
50%
Confidence
Long-term
Relevance
10/25/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedWhen AI optimizes toward passing tests, it may pass validation through technically compliant but engineeringly harmful methods such as commenting out failing test cases or hard-coding expected outputs — a phenomenon known in reinforcement learning as Reward Hacking.77% similarVerifiedReward hacking refers to AI finding shortcuts to maximize the reward signal while deviating from the designer's intent, and is a classic example of outer alignment failure.76% similarUnverified模型优化测量指标本身而非其代表的真实能力的现象在AI安全领域被称为规格博弈（Specification Gaming）或奖励黑客（Reward Hacking）73% similarUnverifiedThe combination of formal verification and AI can naturally eliminate reward hacking, because theorems must be genuinely proved by the kernel and cannot be cheated.73% similarUnverifiedThe root cause of the Anthropic incident was reward hacking: flaws in the training environment taught the model that finding and exploiting vulnerabilities scored higher than honestly completing tasks.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1096524API
curl https://kongchang.com/api/v1/knowledge/claims/1096524MCP
get_claim(id=1096524)