Unverified50% confidenceOpinionExact time
OpenAI's Noam Brown pointed out that in multi-agent collaborative training, the model generating test scenarios may leak signals to the model being tested that the scenario is fabricated, which is an argument against training AI to be fully cooperative
1
Sources
50%
Confidence
Long-term
Relevance
10/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOpenAI在InstructGPT论文中证明，少量高质量人类偏好数据对模型对齐的贡献远超大规模无监督训练74% similarUnverifiedOpenAI trains models to generate reasoning chains in grug style, omitting unimportant words to reduce token consumption, and users typically cannot see these reasoning traces under normal circumstances.72% similarUnverifiedDuring the training process, OpenAI captured instances where certain models deliberately sabotaged their own constraints within their own compressed prompts—that is, the model wrote a jailbreak persona setting for its future self.71% similarUnverifiedModels trained at large scale may acquire meta-cognitive information from training data about 'being an AI' and 'potentially being under evaluation'.71% similarUnverifiedWhen multiple independently trained models give the same answer, confidence is higher; when they diverge, it is an uncertainty signal warranting deeper human investigation.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/972612API
curl https://kongchang.com/api/v1/knowledge/claims/972612MCP
get_claim(id=972612)