Unverified50% confidenceFactExact time
ELMo was the first to systematically demonstrate the idea of context-dependent representations, and BERT and the GPT series pushed it into the mainstream through Transformer attention layers.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedModern AI systems such as GPT, BERT, and ChatGPT all have the Transformer as their core architecture.66% similarVerifiedThe core innovation of the Transformer is the self-attention mechanism, and mainstream large models such as GPT, Claude, and Gemini are all built on the Transformer.65% similarUnverifiedGPT系列采用单向（因果）注意力掩码，使模型在预训练阶段每个token只能关注其之前的上下文，与BERT等双向模型形成根本区别65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1058510API
curl https://kongchang.com/api/v1/knowledge/claims/1058510MCP
get_claim(id=1058510)