Unverified50% confidenceFactExact time
decode阶段的生成速度近似等于内存带宽÷模型大小，例如在内存带宽273GB/s的M4 Pro上运行4GB的Q4量化7B模型理论最大生成速度约68 tokens/s
1
Sources
50%
Confidence
Long-term
Relevance
8/7/2026
First Seen
Sources
Homebench：本地大模型速度、内存与质量三维评测工具详解
Related Claims
UnverifiedQuantprobe claims to enable a machine with only 6GB of VRAM and 16GB of RAM to run 30B-class models at 22 tokens/s.79% similarUnverified自回归推理的decode阶段是纯粹的内存带宽受限操作，将模型从FP16压缩到1-bit理论上可将token生成速度提升16倍78% similarPartially VerifiedFor the 4-bit quantized version of a 7B parameter model, at least 8GB of memory and a modern multi-core CPU are typically required for smooth operation, with inference speeds of approximately 10–30 tokens per second76% similarUnverifiedA 70B parameter model quantized with Q4_K_M occupies approximately 40GB of storage, and can achieve an inference speed of 10–20 tokens per second on an RTX 4090 with CPU memory offloading.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/705467API
curl https://kongchang.com/api/v1/knowledge/claims/705467MCP
get_claim(id=705467)