Unverified50% confidenceFactExact time
For a typical configuration (L layers, hidden dimension d, GQA with h KV heads), the KV Cache size per token is approximately 2 × L × h × d_head × number of bytes.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe KV cache size can be expressed with the formula cache = 2 (K and V) × layers × KV heads × head dim × tokens × 2 bytes, where 2 bytes corresponds to FP16, doubling when using FP32, and halving when using INT8 quantization.77% similarUnverifiedThe formula for estimating the KV cache of a standard full-attention decoder layer is KV bytes = 2 × L × Hkv × D × T × C × S, where 2 stands for keys and values, L is the number of attention layers, Hkv is the number of key/value heads per layer, D is the dimension of each key/value head, T is the number of tokens cached per sequence, C is the number of bytes per cache element, and S is the number of independent sequences.76% similarUnverifiedFor a model with L layers and hidden dimension d, the KV Cache footprint for a single token is 2×L×d×sizeof(float16) bytes.76% similarUnverifiedWith KV Cache, each round of Q projection only needs to compute a 1×D_k vector, whereas without caching, a full N×D_k matrix must be computed.75% similarVerifiedThe size of the KV cache is determined by three multiplied dimensions: the size of each record, how many tokens are combined into one record, and whether layers can share caches with each other.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1062114API
curl https://kongchang.com/api/v1/knowledge/claims/1062114MCP
get_claim(id=1062114)