Unverified50% confidenceOpinionExact time
The core limitation of safety alignment lies in this: alignment training relies on classificatory judgments of "harmful intent," while risks in professional domains such as medicine and law are often hidden in semantically harmless phrasing; what the model learns is "what kind of question looks like a harmful request," not "what kind of answer would cause real harm."
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMost existing offline policy evaluation and safe policy learning tools provide guarantees at the population level and struggle to answer the question of "for whom it works and for whom it harms."79% similarUnverifiedRelying on prompt constraints to prevent dangerous operations is inherently unreliable, because the model may be jailbroken, make misjudgments, or forget constraints during long conversations.78% similarUnverifiedRelying solely on model-side safety training is considered insufficient to address the risks posed by agents in real-world environments78% similarUnverifiedSafe reinforcement learning constrains the probability or cumulative cost of violating safety constraints, rather than solely pursuing robustness to perturbations.78% similarUnverifiedSafety alignment should not only train models to "refuse dangerous requests" but also train them to "recognize what they do not know".78% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1105689API
curl https://kongchang.com/api/v1/knowledge/claims/1105689MCP
get_claim(id=1105689)