Unverified50% confidenceOpinionExact time
Failure-driven PPO fine-tuning may be inferior to plain continuation training, with possible causes including distribution shift, reward-shaping side effects, and curriculum order sensitivity.
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
Verified仅用成功轨迹训练的模型往往存在分布外泛化的严重缺陷，遇到工具返回错误时便丧失纠偏能力75% similarUnverifiedPredictive disagreement arises among independently pretrained PFNs, which is an external manifestation of amortization error.74% similarUnverifiedWhen an LLM serves as a reward judge, the determinism and reproducibility of rewards decrease, and the same answer may receive different scores under different prompts or sampling, posing challenges to training stability.74% similarVerifiedBehavioral cloning simplifies the problem to supervised learning, but suffers from distribution shift, where differences between the state distributions at training time and execution time lead to error accumulation.73% similarUnverifiedUnconstrained recursive self-improvement can lead to performance drift, runaway costs, and even contamination of training data through leakage.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1109982API
curl https://kongchang.com/api/v1/knowledge/claims/1109982MCP
get_claim(id=1109982)