Unverified60% confidenceFactExact time
The original Transformer paper used sinusoidal positional encoding, while subsequent models such as BERT switched to learnable positional embeddings.
2
Sources
60%
Confidence
Long-term
Relevance
7/13/2026
First Seen
Sources
Why Do Transformer Learning Paths Always Start Too Late? Order Matters More Than Materials
redditr/learnmachinelearning7/11/2026
Related Claims
UnverifiedTransformer uses Positional Encoding to retain word order information while performing parallel computation.79% similarUnverified位置编码经历了从固定正弦编码（原始论文）到可学习绝对位置编码（BERT）再到旋转位置编码RoPE（LLaMA、Qwen）的演进77% similarUnverified自注意力本身对顺序不敏感，打乱词语顺序注意力权重计算结果相同，因此需要引入位置编码；原始Transformer论文使用正余弦函数编码74% similarVerifiedThe Transformer's encoder is responsible for mapping the input sequence to a contextual representation and is suitable for classification and understanding tasks, while the decoder autoregressively generates the output sequence and is suitable for text generation tasks74% similarUnverifiedTransformer uses positional encoding based on sine and cosine functions, with wavelengths arranged in a geometric progression, where the shortest and longest differ by a factor of ten thousand.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/496134API
curl https://kongchang.com/api/v1/knowledge/claims/496134MCP
get_claim(id=496134)