Verified80% confidenceFactTime unknown
The attention mechanism of the Transformer architecture significantly reduces its focus on early content when processing very long sequences, causing the model to forget earlier settings
29
Sources
80%
Confidence
Long-term
Relevance
5/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe performance decline caused by 'lost in the middle' follows a U-shaped curve rather than linear decay; its mechanism may be related to Transformer attention allocation and positional encoding.79% similarUnverified当前主流LLM采用Transformer架构，随着上下文长度增加会出现对早期信息的注意力衰减现象79% similarUnverifiedThe computational cost of the Transformer self-attention mechanism grows quadratically with context length, and the "Lost in the Middle" effect causes the model to extract information from the middle of the window less effectively than from the beginning and end.79% similarUnverifiedCurrent large models do not have a true persistent memory mechanism, and as the number of conversation turns increases, the weight of early requirement descriptions in the attention mechanism gradually decays.78% similarVerifiedEven models with a million-token context face an attention dilution problem when handling large codebases—when the context is too long, the model's attention to early content drops significantly.78% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/7312API
curl https://kongchang.com/api/v1/knowledge/claims/7312MCP
get_claim(id=7312)