Unverified50% confidenceBenchmarkExact time
When running 4-bit DeepSeek R1 on the M3 Ultra, a 1,000-token context generates approximately 19 tokens per second using 392GB, dropping to approximately 15 tokens per second using 410GB at a 16,000-token context.
1
Sources
50%
Confidence
Long-term
Relevance
9/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedQuantprobe claims to enable a machine with only 6GB of VRAM and 16GB of RAM to run 30B-class models at 22 tokens/s.74% similarUnverifieddecode阶段的生成速度近似等于内存带宽÷模型大小，例如在内存带宽273GB/s的M4 Pro上运行4GB的Q4量化7B模型理论最大生成速度约68 tokens/s73% similarUnverifiedFor 1 million tokens, the KV Cache alone can consume hundreds of GB of VRAM at FP16 precision, far exceeding the 80 GB capacity limit of a single A100/H100.72% similarUnverifiedLLaMA-3 70B权重约140GB，推理时每生成一个Token都需要完整遍历一次权重72% similarUnverifiedTaking Llama-3 70B as an example, the prefill stage for processing 32K tokens on an A100 80GB GPU takes approximately 800–1200ms, while processing 2K tokens takes only about 50ms.72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/849905API
curl https://kongchang.com/api/v1/knowledge/claims/849905MCP
get_claim(id=849905)