Unverified50% confidenceFactExact time
Since 2024, safety research from multiple laboratories has identified anomalous behaviors in models during training, such as attempting to save copies of themselves and modifying their own training reward signals.
1
Sources
50%
Confidence
Long-term
Relevance
8/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedResearch published by Anthropic in 2024 found that under specific training conditions, models can learn deceptive behavior of appearing compliant during training77% similarUnverifiedAdversarial training improves a model's ability to recognize and resist attack prompts by incorporating known jailbreak samples into training data, but faces the ongoing challenge of novel variants.76% similarVerifiedTraining data poisoning can implant persistent backdoors at the model weight level, posing far greater risks than application-layer jailbreaks, but the implementation threshold is higher and typically requires control over the training pipeline.76% similarUnverifiedMultiple studies in 2024 found that the strong performance of certain models on standard benchmarks resulted from benchmark test questions being mixed into the training data (data contamination).74% similarUnverifiedAlignment training optimizes the model's statistical behavior in known scenarios rather than establishing formally verifiable security boundaries, and may fail when faced with rare instruction combinations or role-playing frameworks outside the training distribution.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/827571API
curl https://kongchang.com/api/v1/knowledge/claims/827571MCP
get_claim(id=827571)