Unverified50% confidenceFactExact time
For million-token-level contexts, if each layer independently maintains a complete KV cache, the GPU memory overhead often exceeds the model weights themselves.
1
Sources
50%
Confidence
Long-term
Relevance
9/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTraditional KV cache implementations pre-allocate contiguous GPU memory blocks of maximum length, resulting in an average of 60%–80% of pre-allocated memory being wasted.70% similarUnverifiedPer-CPU map can avoid lock contention when multiple cores concurrently access the same cache entry, at the cost of increased memory usage and a trade-off in cross-core data consistency69% similarUnverifiedHigher occupancy is not always better; excessively pursuing occupancy may squeeze the registers and shared memory available to each thread, actually slowing down execution.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/921487API
curl https://kongchang.com/api/v1/knowledge/claims/921487MCP
get_claim(id=921487)