Bounded Replay
An optimization mechanism used by inference engines when processing sliding-window attention layers. KV cache is retained only for the limited history inside the window, avoiding wasted GPU memory on tokens outside the window and reducing redundant computation during the prefill stage.
Timeline (last 90 days)
DeepSeek's bounded replay keeps active conversation memory only in local RAM for about three minutes, and after the RAM is cleared the server re-reads only the last roughly 140 words.
The sliding window with bounded replay only replays the most recent 128 tokens, accepting approximation errors to solve the cache offloading problem.
All Facts (2)
DeepSeek's bounded replay keeps active conversation memory only in local RAM for about three minutes, and after the RAM is cleared the server re-reads only the last roughly 140 words.
50%UnverifiedThe sliding window with bounded replay only replays the most recent 128 tokens, accepting approximation errors to solve the cache offloading problem.
50%