Unverified50% confidenceBenchmarkExact time
In terms of prompt processing speed, the RTX 4090 achieves approximately 3,000 tokens/s and the M5 Max approximately 800 tokens/s, giving NVIDIA a clear advantage in long-document and codebase input scenarios.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/4/2026
First Seen
Valid until: 1/2/2027
Sources
Related Entities
Related Claims
UnverifiedWhen VRAM is sufficient and the model is fully resident on the GPU, an RTX 4090 running a Q4-quantized 27B model can typically achieve 15–30 tokens/s; once CPU offloading is triggered, the speed may drop sharply to 1–2 tokens/s or even lower.79% similarVerifiedOn the RTX 5090, Muse Glimmer's generation speed increases from 75 tokens per second to 233 tokens per second, a speedup of approximately 3.1×.75% similarUnverifiedThe RTX 4090 has a theoretical FP16 compute performance of 330 TFLOPS, and the actual Decode speed for a 7B FP16 model is typically 50–60 tokens/s.72% similarUnverifiedOn an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.71% similarUnverifiedOn high-performance laptops, quantized Gemma models can achieve token generation speeds of 20–50 tokens per second; pure CPU inference is approximately 5–15 tokens/second, and INT4 quantization on NVIDIA RTX 4060/4070 can boost inference speed by 3–5 times.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/973167API
curl https://kongchang.com/api/v1/knowledge/claims/973167MCP
get_claim(id=973167)