SmoothQuant
A method for reducing quantization error in large language models by migrating quantization difficulty from activations to weights. It applies mathematically equivalent transformations to smooth activation outliers, improving accuracy under low-bit quantization.
Timeline (last 90 days)
Methods such as SmoothQuant and AWQ mitigate quantization accuracy loss by applying scaling migration to outlier channels or protecting them with mixed precision.
Low-bit representation significantly amplifies the impact of outliers in weight distributions; SmoothQuant addresses this by migrating quantization difficulty, while SpQR addresses it by separately retaining high precision for outliers.
SmoothQuant (2022) smoothly transfers outliers in activations to weights through a mathematically equivalent transformation, enabling both activations and weights to be quantized using INT8.
Techniques such as GPTQ, AWQ, and SmoothQuant enable large models to maintain performance close to their original precision under INT4 quantization.
All Facts (4)
Methods such as SmoothQuant and AWQ mitigate quantization accuracy loss by applying scaling migration to outlier channels or protecting them with mixed precision.
50%UnverifiedLow-bit representation significantly amplifies the impact of outliers in weight distributions; SmoothQuant addresses this by migrating quantization difficulty, while SpQR addresses it by separately retaining high precision for outliers.
50%UnverifiedSmoothQuant (2022) smoothly transfers outliers in activations to weights through a mathematically equivalent transformation, enabling both activations and weights to be quantized using INT8.
50%UnverifiedTechniques such as GPTQ, AWQ, and SmoothQuant enable large models to maintain performance close to their original precision under INT4 quantization.
50%