Concept
NVFP4 KV Cache
A technique for storing the key-value cache (KV cache) in the low-precision NVFP4 format, intended to reduce memory usage during long-context inference. It typically requires accompanying scaling strategies to maintain accuracy.
Timeline (last 90 days)
Oct 29
vLLM integrated three serving-stack configurations for M3.1 Flash: native NVFP4 KV cache, NVFP4 MoE weights, and DSpark speculative decoding.
Unverified50%