KVQuant

A KV cache quantization method for autoregressive LLMs that introduces sensitivity-weighted non-uniform quantization, outlier handling, and calibration data to reduce accuracy loss at low bit widths.