DCKV
An end-to-end differentiable KV cache codec that achieves high-ratio KV cache compression by jointly optimizing the transform, quantizer, and entropy model together with a frozen language model.
Timeline (last 90 days)
DCKV is a differentiable KV cache codec that, for the first time, enables end-to-end training, jointly optimizing the transform, quantizer and entropy model with a frozen language model.
DCKV's training objective contains two terms: the code length allocated by the entropy model (compression rate) and the change in the model's output (the distortion introduced by compression).
DCKV's entropy model is conditioned on prompt tokens and the already-decoded adjacent positions, and the decoding process is autoregressive.
On the Qwen3.5-2B model, DCKV reduces the number of stored bytes per compressed token to more than 55 times lower than bf16, while accuracy loss on three long-context benchmarks does not exceed 1.8 percentage points.
When the compression rate is further increased to more than 100 times, DCKV's accuracy loss stays within 3.6 percentage points.
DCKV is still at the research stage; its encoding and decoding process itself requires additional computation, and the autoregressive nature of entropy coding may introduce decoding latency.
DCKV is designed for hybrid language models, and its applicability to pure attention architectures remains to be verified.
All Facts (7)
DCKV is a differentiable KV cache codec that, for the first time, enables end-to-end training, jointly optimizing the transform, quantizer and entropy model with a frozen language model.
50%UnverifiedDCKV's training objective contains two terms: the code length allocated by the entropy model (compression rate) and the change in the model's output (the distortion introduced by compression).
50%UnverifiedDCKV's entropy model is conditioned on prompt tokens and the already-decoded adjacent positions, and the decoding process is autoregressive.
50%UnverifiedOn the Qwen3.5-2B model, DCKV reduces the number of stored bytes per compressed token to more than 55 times lower than bf16, while accuracy loss on three long-context benchmarks does not exceed 1.8 percentage points.
50%UnverifiedWhen the compression rate is further increased to more than 100 times, DCKV's accuracy loss stays within 3.6 percentage points.
50%UnverifiedDCKV is still at the research stage; its encoding and decoding process itself requires additional computation, and the autoregressive nature of entropy coding may introduce decoding latency.
50%UnverifiedDCKV is designed for hybrid language models, and its applicability to pure attention architectures remains to be verified.
50%