Expired50% confidenceFactTime unknown
Unlike RNNs, the Transformer does not process sequences step by step and therefore lacks an inherent sense of order, requiring positional information to be injected through Positional Encoding
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 9/30/2026(expired)
Sources
Demystifying Transformer: A Word-Continuation Function, Deconstructed
bilibiliAI算法工程师Fiona
Related Claims
UnverifiedPositional Encoding assigns a unique identifier to each position via sinusoidal functions or learnable embedding vectors, compensating for the Transformer's lack of inherent sequential order perception.83% similarUnverifiedTransformer uses Positional Encoding to retain word order information while performing parallel computation.83% similarUnverifiedThe original Transformer paper used sinusoidal positional encoding, while subsequent models such as BERT switched to learnable positional embeddings.78% similarUnverified自注意力本身对顺序不敏感，打乱词语顺序注意力权重计算结果相同，因此需要引入位置编码；原始Transformer论文使用正余弦函数编码78% similarVerifiedBefore the advent of Transformer, the field of natural language processing relied primarily on RNNs and LSTMs to process sequential data, architectures that had to process text word by word in sequence and could not perform parallel computation75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/54652API
curl https://kongchang.com/api/v1/knowledge/claims/54652MCP
get_claim(id=54652)