Unverified50% confidenceOpinionExact time
John Schulman explained that when doing AI research one usually does not directly optimize the objective one ultimately cares about, because obtaining feedback is extremely expensive and infrequent, and labs currently rely heavily on human intuition.
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core scenario that alignment researchers worry about is: sub-goals generated by an agent to achieve broad objectives may systematically deviate from true human intent, with errors cascading and spreading before being discovered.73% similarUnverifiedWhen the research process is highly encapsulated within an AI system, if the reasoning process is a black box to humans, the reliability, interpretability, and credibility of conclusions will be greatly diminished.73% similarUnverifiedCurrent mainstream AI scientist evaluation benchmarks almost exclusively focus on final outputs while discarding the reasoning process.73% similarUnverifiedThe reason frontier labs hesitate about consumer-grade AI is not insufficient technology, but the unfavorable economics behind it.72% similarUnverifiedCurrent AI-assisted research still requires human researchers to oversee decision-making, has not yet broken free from human control to form a closed-loop acceleration, and true RSI remains a theoretical hypothesis.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1012996API
curl https://kongchang.com/api/v1/knowledge/claims/1012996MCP
get_claim(id=1012996)