Unverified60% confidenceFactExact time
Double quantization further quantizes the FP32 scaling factors shared among every 64 parameters into FP8 format, reducing the quantization overhead per parameter from 0.5 bits to approximately 0.127 bits.
2
Sources
60%
Confidence
Long-term
Relevance
8/26/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedA 140 GB FP16 70B model can be compressed to approximately 35–40 GB after 4-bit quantization, reducing communication overhead by about 75%; 2-bit quantization can compress it to approximately 18 GB but introduces accuracy loss.78% similarUnverifiedTaking the Q4_0 format as an example, every 32 FP32 weights are packed into one quantization block containing one scale factor and 32 4-bit integers, averaging approximately 4.5 bits per parameter, reducing memory usage to approximately 1/8 of the original.77% similarUnverifiedQuantization converts FP32 weights to INT8 or FP16, reducing computation and memory bandwidth pressure while maintaining audio quality.76% similarVerifiedFP8 quantization can compress the weights of a 7B model from approximately 14GB (FP16) to approximately 7GB.75% similarUnverifiedA 2.4-trillion-parameter model requires approximately 9.6 TB of VRAM in FP32, about 4.8 TB in BF16, about 2.4 TB in INT8, and about 1.2 TB in INT4.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/802104API
curl https://kongchang.com/api/v1/knowledge/claims/802104MCP
get_claim(id=802104)