Unverified50% confidenceSolutionExact time
Strata adopts hierarchical KV cache management, keeping only 32K units of KV cache in VRAM while offloading the rest to memory; in actual testing the KV VRAM hit rate reached 98.9%.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
UnverifiedWhen the context exceeds 64,000 tokens, Strata keeps most of the KV cache in memory and retains only the hottest 32,000 tokens in VRAM, improving speed from approximately 51 to 63 tokens/second at a 262,000-token context.77% similarUnverifiedTraditional KV cache implementations pre-allocate contiguous GPU memory blocks of maximum length, resulting in an average of 60%–80% of pre-allocated memory being wasted.72% similarVerifiedQuantized KV caches (such as storing KV vectors in FP8 or INT4) can compress the cache size to 1/2 or even 1/4 of the original.72% similarUnverifiedvLLM's PagedAttention splits the KV cache into fixed-size physical blocks (typically 16 tokens), improving GPU memory utilization from below 40% to close to 90%.71% similarUnverified传统LLM推理中，KV Cache的内存管理导致60%-80%的GPU显存被浪费71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/978773API
curl https://kongchang.com/api/v1/knowledge/claims/978773MCP
get_claim(id=978773)