Unverified50% confidenceBenchmarkExact time
The RTX 4090 has a theoretical FP16 compute performance of 330 TFLOPS, and the actual Decode speed for a 7B FP16 model is typically 50–60 tokens/s.
1
Sources
50%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAt FP16 half-precision computation, the RTX 4090 has a theoretical performance of approximately 330 TFLOPS, while the M2 Ultra's GPU delivers approximately 27.2 TFLOPS.79% similarUnverifiedWhen VRAM is sufficient and the model is fully resident on the GPU, an RTX 4090 running a Q4-quantized 27B model can typically achieve 15–30 tokens/s; once CPU offloading is triggered, the speed may drop sharply to 1–2 tokens/s or even lower.78% similarUnverifiedThe model can achieve approximately 206 tokens/second on a single RTX 5090 using the NVFP4 scheme.75% similarUnverifiedRTX 4090算力约165 TFLOPS，而单路推理实际只需约54 TFLOPS，算力严重过剩74% similarUnverifiedA single TPU v4 chip has a peak compute of approximately 275 TFLOPS (BF16 precision), and a TPU v4-32 pod consists of 32 chips interconnected via ICI high-speed interconnects, with a total compute of approximately 8,800 TFLOPS.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/887580API
curl https://kongchang.com/api/v1/knowledge/claims/887580MCP
get_claim(id=887580)