Product
AttentionStore
A system for persisting and managing attention KV caches in large-model inference, supporting cross-tier storage across GPU, CPU, and SSD to extend KV-cache capacity for long-session and multi-turn dialogue scenarios.
Timeline (last 90 days)
Sep 16
Systems such as Mooncake, LMCache, FlexGen, InfiniGen, and AttentionStore attempt to extend GPU VRAM capacity using CPU DRAM and SSDs.
Unverified50%