LRU

Modern databases such as MySQL InnoDB and PostgreSQL both implement a buffer pool mechanism internally, managing cache replacement strategies through algorithms such as LRU.

The MoE cache groups expert tensors by data type and shape, allocating one shared slot pool per group in VRAM and using an LRU eviction policy.

Any combination of bounded storage plus an eviction policy implicitly contains a value judgment about "what is worth keeping"; the default LRU or FIFO presupposes that "time" is a proxy variable for importance, but this fails in scenarios such as retries, deduplication, and anomaly detection.

Both LRU (least recently used) and FIFO (first in, first out) eviction policies treat "time" as a proxy variable for importance, but in the debug capturer scenario, observation frequency is negatively correlated with information value—what appears at high frequency is retry noise, while the malformed payload that appears only once at low frequency is what is actually wanted.

Cache eviction policies include LRU, LFU, and weighted schemes that consider prefix length or recomputation cost; frequent cache eviction reduces the effectiveness of cache-aware routing.

GPU expert weight caching typically uses a replacement strategy similar to LRU (Least Recently Used), prioritizing retention of expert weights that have been frequently invoked recently.

LRU (Least Recently Used) is a cache eviction algorithm whose core idea is to evict the data that has not been accessed for the longest time.

LRU does not rely on predictions of future access patterns and incurs almost no additional computational overhead.

LRU is far harder to beat than KV-Cache papers suggest.

A large body of research has attempted to design smarter cache eviction strategies than LRU, such as deciding which cache entries to retain based on token importance, attention scores, or access frequency.

Advanced caching strategies require additional metadata tracking, scoring computation, and maintenance costs, and after accounting for these overheads, their net gain relative to LRU may shrink substantially.

LRU is extremely lightweight to implement and performs stably across various access patterns, whereas the advantages of complex algorithms in specific patterns are often accompanied by performance degradation in other patterns.

Fair evaluation of caching algorithms requires testing on a broad and representative set of workloads, with a well-tuned LRU serving as the strong baseline.

For teams building large model inference services, a well-implemented and well-tuned LRU is often already a quite powerful choice, and alternative solutions should be comprehensively tested on actual workloads before a decision is made.

The classic LRU algorithm is prone to cache pollution in one-time large-batch read scenarios such as full-table scans, evicting hot data from the cache.

Using an LRU (Least Recently Used) caching strategy allows hot site data to be cached in memory while cold site data falls back to disk or a database.

The LRU cache replacement policy requires at least 24 bytes of metadata overhead per entry on 64-bit systems (16 bytes for a doubly linked list + 8 bytes for a timestamp).

Redis的LRU实现并非教科书中严格的链表方案，而是采用近似LRU算法（随机采样+淘汰池）

Orvieto等人于2022年提出了Linear Recurrent Unit（LRU），使用对角复数矩阵作为循环核心