Unverified50% confidenceFactExact time
A 70B-parameter model (140 GB of weights) has a theoretical maximum generation speed of approximately 7 tokens/second at 1 TB/s bandwidth.
1
Sources
50%
Confidence
Long-term
Relevance
9/7/2026
First Seen
Sources
Related Claims
UnverifiedIf the quantized model weights are 150 GB, reading the weights once at a bandwidth of 273 GB/s takes approximately 0.55 seconds, giving a theoretical single-token generation speed of approximately 1.8 tokens/s (excluding computation latency and cross-node communication overhead).80% similarUnverified128GB DDR5 memory can fully load all weights of a 70B-class model, but at a speed of approximately 2–5 tokens/s.77% similarUnverifiedQuantprobe claims to enable a machine with only 6GB of VRAM and 16GB of RAM to run 30B-class models at 22 tokens/s.75% similarUnverifiedThe M2 Max (96 GB unified memory) can theoretically run quantized models at the 70B level smoothly.74% similarPartially VerifiedFor the 4-bit quantized version of a 7B parameter model, at least 8GB of memory and a modern multi-core CPU are typically required for smooth operation, with inference speeds of approximately 10–30 tokens per second74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/870301API
curl https://kongchang.com/api/v1/knowledge/claims/870301MCP
get_claim(id=870301)