Expired60% confidenceFactTime unknown
BERT uses Masked Language Modeling as its pre-training task, while GPT uses Causal Language Modeling (predicting the next word) as its pre-training task.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 9/30/2026(expired)
Sources
Related Claims
UnverifiedIn ChatGPT's training, the pre-training phase uses unsupervised learning for next-token prediction, the instruction fine-tuning phase introduces supervised learning, and finally RLHF is used to align the model's outputs.75% similarUnverifiedThe BERT pre-trained model was introduced in 2018; by pre-training on massive amounts of text, the model is able to capture subtle differences in the meaning of words across different contexts.73% similarVerified大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律72% similarUnverifiedBERT采用双向编码器架构，通过掩码语言模型预训练，随机遮盖15%的token并预测它们72% similarUnverifiedThe core working principle of large language models such as ChatGPT and Claude is next-token prediction based on massive training data.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/61141API
curl https://kongchang.com/api/v1/knowledge/claims/61141MCP
get_claim(id=61141)