Unverified50% confidenceFactExact time
RLHF faces challenges such as reward hacking, inconsistent human preference annotations, and training instability, which are among the motivations for Anthropic proposing Constitutional AI as a complementary approach.
1
Sources
50%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedRLHF存在奖励黑客行为等根本性局限，当AI能力超越评估员水平时人类评估的可靠性也会下降76% similarUnverified当前的RLHF和Constitutional AI等技术主要解决行为对齐层面的问题，更深层的意图和动机对齐仍是开放性挑战76% similarVerifiedAI sycophancy originates from the Reinforcement Learning from Human Feedback (RLHF) training phase, where human annotators tend to give higher scores to friendly and affirmative responses71% similarUnverifiedAI model over-refusal is a known issue in safety alignment research, where RLHF and Constitutional AI training cause models to incorrectly classify harmless requests as requiring refusal71% similarUnverifiedThe core methods of safety alignment are Reinforcement Learning from Human Feedback (RLHF) and Constitutional AI69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/887181API
curl https://kongchang.com/api/v1/knowledge/claims/887181MCP
get_claim(id=887181)