Unverified60% confidenceBenchmarkExact time
Quantprobe claims to enable a machine with only 6GB of VRAM and 16GB of RAM to run 30B-class models at 22 tokens/s.
2
Sources
60%
Confidence
Medium-term (~90 days)
Relevance
8/5/2026
First Seen
Valid until: 11/3/2026
Sources
Related Claims
VerifiedUnder the M5 Ultra's 1.2 TB/s bandwidth, the theoretical speed of a 100 GB quantized model is approximately 12 tokens/s, a 200 GB model approximately 6 tokens/s, and a 300 GB model approximately 4 tokens/s.79% similarUnverifieddecode阶段的生成速度近似等于内存带宽÷模型大小，例如在内存带宽273GB/s的M4 Pro上运行4GB的Q4量化7B模型理论最大生成速度约68 tokens/s79% similarUnverifiedIf a system has only 16 GB or 32 GB of system memory, the complete inference process may not be possible even with the same GPU model.78% similarUnverified48GB of memory can fit a 4-bit quantized 32B model, but the inference speed may slow to fewer than 1 token per second, making it nearly unusable for interactive purposes78% similarUnverifiedTesting with a 4-bit quantized Qwen2.5 7B model on an M2 Mac mini with 16 GB unified memory, throughput dropped from 16.8 tokens/s without batching to 8.7 tokens/s with batching.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/689090API
curl https://kongchang.com/api/v1/knowledge/claims/689090MCP
get_claim(id=689090)