PTQ
A quantization technique that directly compresses model weights numerically after training is complete. It is straightforward to apply but incurs greater accuracy degradation at low precision compared to QAT.
Core Facts
Post-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
PTQ (Post-Training Quantization) does not require retraining the model; quantization can be completed with only a small amount of calibration data, typically a few hundred to a few thousand calibration samples.
Timeline (last 90 days)
BitLens currently focuses on PTQ scenarios and depends on the representativeness of calibration data; the Omega and kurtosis metrics can efficiently flag fragile layers but cannot replace end-to-end accuracy validation.
In practice QAT usually outperforms PTQ, especially at low bit widths such as 4-bit.
Post-training quantization (PTQ) can usually still maintain quality at 8 bits or even 4 bits, but when pushed to around 2 bits the error increases noticeably.
Post-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
Qwen3 27B is trained in BF16, and forcibly applying 1-bit PTQ to it causes a large amount of numerical information to be discarded, severely impairing the model's internal representational capacity.
Q4 is currently the gold standard for balancing quality and VRAM in local deployment of large models, and 1-bit PTQ is not currently usable in production environments.
All Facts (8)
Post-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
80%VerifiedPTQ (Post-Training Quantization) does not require retraining the model; quantization can be completed with only a small amount of calibration data, typically a few hundred to a few thousand calibration samples.
75%Unverified量化感知训练（QAT）在训练过程中模拟量化误差反向传播，效果通常优于训练后量化（PTQ）
60%UnverifiedBitLens currently focuses on PTQ scenarios and depends on the representativeness of calibration data; the Omega and kurtosis metrics can efficiently flag fragile layers but cannot replace end-to-end accuracy validation.
50%UnverifiedIn practice QAT usually outperforms PTQ, especially at low bit widths such as 4-bit.
50%UnverifiedPost-training quantization (PTQ) can usually still maintain quality at 8 bits or even 4 bits, but when pushed to around 2 bits the error increases noticeably.
50%UnverifiedQwen3 27B is trained in BF16, and forcibly applying 1-bit PTQ to it causes a large amount of numerical information to be discarded, severely impairing the model's internal representational capacity.
50%UnverifiedQ4 is currently the gold standard for balancing quality and VRAM in local deployment of large models, and 1-bit PTQ is not currently usable in production environments.
50%