Unverified50% confidenceFactExact time
For Transformers, mechanisms such as positional encoding and attention masking may fail beyond the training length.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
Unverified标准Transformer在处理需要精确计数、栈操作或复杂状态依赖的任务时表现不稳定，与其注意力机制的软寻址特性有关79% similarUnverifiedThe performance decline caused by 'lost in the middle' follows a U-shaped curve rather than linear decay; its mechanism may be related to Transformer attention allocation and positional encoding.77% similarUnverifiedTransformer模型存在位置偏差，对中间位置信息的检索能力尤其薄弱，即lost in the middle问题74% similarUnverifiedTransformer 中任意两个位置之间的信息传递路径长度恒为1，从根本上克服了 RNN 在长序列上因梯度消失导致的遗忘问题73% similarUnverifiedWhen a Transformer outputs tokens through sampling, it permanently loses information about which possibilities the model considered and their respective confidence levels (a lossy process).73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1103036API
curl https://kongchang.com/api/v1/knowledge/claims/1103036MCP
get_claim(id=1103036)