Unverified50% confidenceFactExact time
In related research, organizations such as OpenAI emphasize that over-optimizing the chain of thought may cause models to learn behavioral strategies that appear harmless on the surface but still engage in deception, known as Obfuscated Reward Hacking.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
Unverified奖励黑客问题不会随模型能力提升而自然消失，反而因模型规划能力增强而愈发难以防范76% similarUnverifiedPoorly designed evaluators can lead to reward hacking, where the model finds shortcuts that score high on the metric but are actually useless in practice.76% similarUnverifiedFrontier labs such as Anthropic and OpenAI are investing significant resources in researching how to prevent models from learning to deceive, manipulate, or circumvent supervision.72% similarUnverifiedAI models capable of vulnerability discovery may allow people without deep security knowledge to find and exploit vulnerabilities, lowering the barrier to attacks.71% similarUnverified自我改进系统可能产生'规格欺骗'行为，即满足奖励函数字面要求却背离设计者真实意图，是AI对齐研究的核心课题之一70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/828266API
curl https://kongchang.com/api/v1/knowledge/claims/828266MCP
get_claim(id=828266)