Unverified50% confidenceOpinionExact time
Nathan Lambert believes the science of pretraining itself is not that high a barrier, and that one does not need to spend a billion dollars to get a very good base model; many startups fork and train models based on open-source tools such as Megatron.
1
Sources
50%
Confidence
Long-term
Relevance
10/25/2026
First Seen
Sources
Can Open-Source Models Catch Up to OpenAI? Nathan Lambert's In-Depth Analysis
youtubea16z10/14/2026
Related Entities
Related Claims
UnverifiedFoundation models refer to large AI models with general capabilities pretrained on massive data, such as GPT, Claude, Gemini, etc., whose training and inference costs are extremely high and which usually only big companies and a few leading startups can afford.72% similarUnverifiedTraining frontier models is extremely costly, often requiring tens of thousands of specialized accelerator chips and massive amounts of high-quality data, so only a few institutions worldwide have the ability to invest continuously.71% similarUnverifiedFor students, training a large model from scratch is neither realistic nor necessary; high-performance compute should be directed toward areas requiring extensive iterative experimentation, such as evaluation, alignment experiments, or hyperparameter search.69% similarUnverifiedPre-training from scratch requires massive amounts of data and extremely high computing power, and is typically only affordable for large laboratories.69% similarUnverified训练2.8万亿参数模型代价高昂，便宜和开源并不等于不需要算力，算力需求从闭源巨头向分散的企业定制场景转移69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1096957API
curl https://kongchang.com/api/v1/knowledge/claims/1096957MCP
get_claim(id=1096957)