dLLM-KVQ
An asymmetric KV cache quantization method for diffusion language models. It selects attention-sensitive K cache entries per layer and retains them at high precision, while applying standard low-bit quantization to the V cache.
Timeline (last 90 days)
dLLM-KVQ uses the trajectories generated by TRACE to select, based on output-level and attention-level behavioral metrics, the K cache entries in each layer that need to be kept at high precision, while applying standard low-bit quantization to the Value cache.
The authors adapt KIVI and KVQuant to Fast-dLLM-style prefix caching and dual cache reuse scenarios, and propose two new methods, TRACE and dLLM-KVQ.
On the LLaDA and Dream models, evaluated on four benchmarks—GSM8K, HumanEval, MATH, and MBPP—dLLM-KVQ achieved the highest average score among all quantization methods in the reported 4-bit and 2-bit prefix/dual cache settings.
Quantization schemes designed for dLLM cache reuse error (such as dLLM-KVQ) are more effective than directly transferring autoregressive quantization methods.
TRACE's trajectory calibration idea and dLLM-KVQ's layer-wise mixed-precision strategy provide a reference direction for running diffusion language models under limited GPU memory.
All Facts (5)
dLLM-KVQ uses the trajectories generated by TRACE to select, based on output-level and attention-level behavioral metrics, the K cache entries in each layer that need to be kept at high precision, while applying standard low-bit quantization to the Value cache.
50%UnverifiedThe authors adapt KIVI and KVQuant to Fast-dLLM-style prefix caching and dual cache reuse scenarios, and propose two new methods, TRACE and dLLM-KVQ.
50%UnverifiedOn the LLaDA and Dream models, evaluated on four benchmarks—GSM8K, HumanEval, MATH, and MBPP—dLLM-KVQ achieved the highest average score among all quantization methods in the reported 4-bit and 2-bit prefix/dual cache settings.
50%UnverifiedQuantization schemes designed for dLLM cache reuse error (such as dLLM-KVQ) are more effective than directly transferring autoregressive quantization methods.
50%UnverifiedTRACE's trajectory calibration idea and dLLM-KVQ's layer-wise mixed-precision strategy provide a reference direction for running diffusion language models under limited GPU memory.
50%