Verified80% confidenceFactTime unknown
大语言模型的预训练阶段本质上属于自监督学习，是一种特殊的非监督学习，模型通过预测下一个词来从海量文本中自动学习语言规律
7
Sources
80%
Confidence
Long-term
Relevance
5/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedModern multilingual large models establish language-agnostic semantic representations during the pre-training phase, building cross-lingual semantic alignment in high-dimensional vector space through joint training on massive multilingual corpora.80% similarVerifiedThe training objective of a Base LLM is to predict the next word, using a paradigm called Autoregressive Language Modeling79% similarUnverifiedThe BERT pre-trained model was introduced in 2018; by pre-training on massive amounts of text, the model is able to capture subtle differences in the meaning of words across different contexts.79% similarUnverifiedIn-context learning was first observed in large language models, and its capability fundamentally stems from the diversity of pretraining data.79% similarUnverifiedLLMs acquire language capabilities by performing unsupervised pre-training on massive text corpora (predicting the next token).79% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/4241API
curl https://kongchang.com/api/v1/knowledge/claims/4241MCP
get_claim(id=4241)