Fast-dLLM
An inference acceleration scheme for diffusion language models that speeds up generation by reusing KV states and prefix caches across denoising steps.
Timeline (last 90 days)
Fast-dLLM inference reuses KV states across denoising steps to accelerate generation, but because bidirectional token representations change as masked tokens are progressively revealed, the reused cache may already deviate from the result of full recomputation.
The authors adapt KIVI and KVQuant to Fast-dLLM-style prefix caching and dual cache reuse scenarios, and propose two new methods, TRACE and dLLM-KVQ.
All Facts (2)
Fast-dLLM inference reuses KV states across denoising steps to accelerate generation, but because bidirectional token representations change as masked tokens are progressively revealed, the reused cache may already deviate from the result of full recomputation.
50%UnverifiedThe authors adapt KIVI and KVQuant to Fast-dLLM-style prefix caching and dual cache reuse scenarios, and propose two new methods, TRACE and dLLM-KVQ.
50%