Mixed-Precision Quantization
A quantization approach that assigns different bit widths to different parts of the same model. Its key challenge lies in measuring the importance of each part in order to allocate the bit budget.
Timeline (last 90 days)
dLLM-KVQ uses the trajectories generated by TRACE to select, based on output-level and attention-level behavioral metrics, the K cache entries in each layer that need to be kept at high precision, while applying standard low-bit quantization to the Value cache.
GGA allocates bit widths to different heads under a model-level mixed-precision budget by evaluating the gradient impact of each attention head's candidate quantization residual on the future prediction loss.
Traditional mixed-precision quantization relies on expert importance ranking, implicitly assuming that assigned weights are linearly related to ranking positions, but ranking can only provide relative order and cannot calibrate the absolute strength of error weights.
The core insight of the paper is that quantization allocation should not stop at sensitivity ranking, but should shift toward objective-aware allocation, that is, reverse-calibrating each expert's error weight according to the final optimization objective.
Mixed-precision quantization applies aggressive low precision such as INT4 to backbone networks that have large parameter counts but are not precision-sensitive, while retaining INT8 or FP16 for precision-sensitive components such as text encoders and QKV projections.
All Facts (5)
dLLM-KVQ uses the trajectories generated by TRACE to select, based on output-level and attention-level behavioral metrics, the K cache entries in each layer that need to be kept at high precision, while applying standard low-bit quantization to the Value cache.
50%UnverifiedGGA allocates bit widths to different heads under a model-level mixed-precision budget by evaluating the gradient impact of each attention head's candidate quantization residual on the future prediction loss.
50%UnverifiedTraditional mixed-precision quantization relies on expert importance ranking, implicitly assuming that assigned weights are linearly related to ranking positions, but ranking can only provide relative order and cannot calibrate the absolute strength of error weights.
50%UnverifiedThe core insight of the paper is that quantization allocation should not stop at sensitivity ranking, but should shift toward objective-aware allocation, that is, reverse-calibrating each expert's error weight according to the final optimization objective.
50%UnverifiedMixed-precision quantization applies aggressive low precision such as INT4 to backbone networks that have large parameter counts but are not precision-sensitive, while retaining INT8 or FP16 for precision-sensitive components such as text encoders and QKV projections.
50%