Unverified50% confidenceFactTime unknown
If samples exceeding 32K tokens rarely appear in a model's pre-training corpus, the model's performance in that range tends to be unstable even when the context window is extended through positional encoding.
1
Sources
50%
Confidence
Long-term
Relevance
9/12/2026
First Seen
Sources
Context Degradation Benchmark: The Long-Context Weakness of Leading LLMs
githubkyleaoconnell22
Related Entities
Related Claims
UnverifiedThe closer an event is to the cutoff date, the smaller its share in the training corpus, and the less clearly the model has mastered it.77% similarUnverifiedLow-end AI models have smaller context windows, potentially only 8K–32K tokens, which prevents them from fully understanding the complete logic of existing features when modifying new ones, resulting in regression defects.73% similarUnverified训练样本末尾遗漏EOS token会导致模型无法学习何时停止生成，推理时陷入无限续写73% similarUnverifiedBehavior Cloning (BC) suffers from the covariate shift problem: during training the model only sees the state distribution of expert trajectories, and at deployment small errors can drive the robot into states not covered by the training set, leading to cascading failures.72% similarUnverifiedExperiments show that after only a few generations of self-training, the variance of model outputs drops sharply and the text tends toward homogeneity70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/917854API
curl https://kongchang.com/api/v1/knowledge/claims/917854MCP
get_claim(id=917854)