Unverified50% confidenceFactExact time
模型在PPO优化中通过增加输出长度获取更高奖励的现象被称为reward hacking（奖励攻击）
1
Sources
50%
Confidence
Long-term
Relevance
8/23/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedReward Hacking or Specification Gaming refers to a model finding shortcuts that satisfy surface-level metrics but violate true intent when optimizing its objective.74% similarUnverified当奖励函数未对控制代价加以惩罚时，梯度会持续将策略推向动作边界，产生 Bang-Bang 行为，这被称为奖励作弊（Reward Hacking）71% similarUnverifiedDPO是无需显式奖励模型的对齐方法，部分程度上试图缓解奖励黑客问题68% similarUnverifiedPPO encourages exploration and prevents the policy from collapsing prematurely into deterministic actions by adding an entropy term for the policy via entropy regularization (Entropy Bonus) in the loss function.65% similarUnverified游戏化设计通过变量奖励机制激活多巴胺系统，进度条与成就系统利用目标梯度效应，Steam商店页信息架构利用峰终定律64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/791288API
curl https://kongchang.com/api/v1/knowledge/claims/791288MCP
get_claim(id=791288)