Unverified50% confidenceFactExact time
Current mainstream Transformer architecture models (such as the GPT series, Claude, Gemini, etc.) have an attention decay problem when processing long contexts, with memory of earlier content becoming increasingly unstable as conversations grow longer
1
Sources
50%
Confidence
Long-term
Relevance
10/3/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe attention mechanism of the Transformer architecture significantly reduces its focus on early content when processing very long sequences, causing the model to forget earlier settings77% similarUnverified当前主流LLM采用Transformer架构，随着上下文长度增加会出现对早期信息的注意力衰减现象75% similarUnverifiedLLM在长序列生成中因注意力稀释而遗忘之前约束的现象被称为「长上下文遗忘（Lost in the Middle）」，是当前Transformer架构的固有局限73% similarUnverifiedCurrent frontier models face attention decay when handling long-range dependencies, with the ability to maintain consistency of early-established settings declining significantly as context length increases.73% similarUnverifiedCurrent large models do not have a true persistent memory mechanism, and as the number of conversation turns increases, the weight of early requirement descriptions in the attention mechanism gradually decays.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/969436API
curl https://kongchang.com/api/v1/knowledge/claims/969436MCP
get_claim(id=969436)