Context Rot
A phenomenon in which, as the input context grows longer, a large model's accuracy in retrieving and reasoning over information in middle positions declines and the model becomes susceptible to outdated or contradictory information, arising because attention weights are diluted by irrelevant tokens.
Timeline (last 90 days)
The researchers call the phenomenon where "the more tokens in the window, the harder it is for the model to reliably use any of them" "context rot."
The industry uses “Context Rot” to describe the phenomenon in which, as input grows longer, the model's accuracy in retrieving and reasoning about information in middle positions declines.
The on-demand loading mechanism keeps irrelevant instructions out of the working context, preventing context corruption.
All Facts (3)
The researchers call the phenomenon where "the more tokens in the window, the harder it is for the model to reliably use any of them" "context rot."
50%UnverifiedThe industry uses “Context Rot” to describe the phenomenon in which, as input grows longer, the model's accuracy in retrieving and reasoning about information in middle positions declines.
50%UnverifiedThe on-demand loading mechanism keeps irrelevant instructions out of the working context, preventing context corruption.
50%