Unverified50% confidenceOpinionExact time
Nathan Lambert believes there is some basis for viewing distillation as a primary means by which Chinese open-source models catch up, but that this is oversimplified; he notes that post-training has now shifted to being mainly about reinforcement learning environments, and that simply using the strongest model to generate completions for SFT and watching scores rise is hard to reproduce in the open literature.
1
Sources
50%
Confidence
Long-term
Relevance
10/25/2026
First Seen
Sources
Can Open-Source Models Catch Up to OpenAI? Nathan Lambert's In-Depth Analysis
youtubea16z10/14/2026
Related Entities
Related Claims
UnverifiedThe author speculates that the opinionated tendency observed in open-source models may be related to distillation training, as the training data comes from commercial large model outputs that are assumed to be correct by default69% similarUnverifiedNathan Lambert believes the science of pretraining itself is not that high a barrier, and that one does not need to spend a billion dollars to get a very good base model; many startups fork and train models based on open-source tools such as Megatron.67% similarUnverifiedWarmup策略在训练初期使用极小学习率,待参数进入合理范围后再提升,被大型语言模型训练广泛采用67% similarUnverifiedLiu believes that neural simulators are more suitable in the early stages for policy validation rather than for directly training policies.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1096950API
curl https://kongchang.com/api/v1/knowledge/claims/1096950MCP
get_claim(id=1096950)