Expired65% confidenceBenchmarkExact time
A 70B parameter model quantized with Q4_K_M occupies approximately 40GB of storage, and can achieve an inference speed of 10–20 tokens per second on an RTX 4090 with CPU memory offloading.
3
Sources
65%
Confidence
Medium-term (~90 days)
Relevance
7/21/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/19/2026(expired)
Sources
Related Claims
UnverifiedQuantization can compress the storage size of a 70-billion-parameter model from approximately 140 GB in FP16 to approximately 35–40 GB in INT4, enabling it to run on consumer-grade hardware such as a single RTX 4090 with CPU memory offloading.84% similarVerifiedQuantization formats can compress the runtime memory of a 70B-parameter model from 140 GB to under 40 GB.79% similarVerifiedEven after quantization, 70B-level models often require more than 40 GB of VRAM or a multi-GPU configuration.76% similarVerifiedThe KV Cache of a 70B-parameter model with 128K context support can require hundreds of gigabytes of VRAM at full load, exceeding the 80GB VRAM capacity of a single H100.76% similarVerifiedA model with 70 billion parameters that originally requires 80 GB of VRAM requires only about 40 GB of memory after 4-bit quantization (Q4_K_M level), with inference quality loss typically within 3%–8% on most benchmarks.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/579919API
curl https://kongchang.com/api/v1/knowledge/claims/579919MCP
get_claim(id=579919)