Unverified50% confidenceSolutionExact time
使用IQ4_XS配合-fa 1 -ctk q8_0 -ctv q8_0对KV缓存进行q8_0量化后，可加载完整32k上下文，常驻内存约16.6GB
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
8/20/2026
First Seen
Valid until: 11/18/2026
Sources
Related Claims
UnverifiedTaking the Q4_0 format as an example, every 32 FP32 weights are packed into one quantization block containing one scale factor and 32 4-bit integers, averaging approximately 4.5 bits per parameter, reducing memory usage to approximately 1/8 of the original.78% similarUnverifiedFor a model of approximately 160 GB (such as the UD-Q3_K_XL quantized version of Qwen3.5-397B), running the lens requires approximately 20 GB of additional memory.77% similarUnverifiedAfter 4-bit quantization, Qwen-7B can run smoothly on a consumer-grade GPU with 24 GB of VRAM.75% similarUnverifiedQ4_K_M量化文件大小17.77GB，pp2048速度96.8 tok/s，tg128速度11.4 tok/s75% similarUnverifiedLlama 3 70B量化版（Q4精度）仍需约35GB内存75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/778779API
curl https://kongchang.com/api/v1/knowledge/claims/778779MCP
get_claim(id=778779)