Non-uniform quantization dynamically allocates bits according to weight distribution, keeping more precision for high-frequency important weights and compressing long-tail weights to extremely low bits, thereby maintaining small error even at very low average bit width.