Unverified60% confidenceFactExact time
Taking the Q4_0 format as an example, every 32 FP32 weights are packed into one quantization block containing one scale factor and 32 4-bit integers, averaging approximately 4.5 bits per parameter, reducing memory usage to approximately 1/8 of the original.
2
Sources
60%
Confidence
Long-term
Relevance
7/21/2026
First Seen
Sources
Related Claims
UnverifiedIn FP16 format each parameter occupies 2 bytes, while in INT4 format it occupies only 0.5 bytes, theoretically achieving a 4x compression ratio.80% similarUnverifiedA 2.4-trillion-parameter model requires approximately 9.6 TB of VRAM in FP32, about 4.8 TB in BF16, about 2.4 TB in INT8, and about 1.2 TB in INT4.80% similarUnverifiedSome ONNX exports labeled q4 are larger than the q4f16 versions because the embedding table retains 32-bit precision; for example, SmolLM2-360M's q4 is 390MB and q4f16 is 275MB, and the two produce identical outputs.78% similarUnverified使用IQ4_XS配合-fa 1 -ctk q8_0 -ctv q8_0对KV缓存进行q8_0量化后，可加载完整32k上下文，常驻内存约16.6GB78% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/580985API
curl https://kongchang.com/api/v1/knowledge/claims/580985MCP
get_claim(id=580985)