Unverified50% confidenceBenchmarkExact time
The typical generation speed of an NVIDIA RTX 4090 running a similarly scaled Q4 quantized model is in the range of 40–60 TPS
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/6/2026
First Seen
Valid until: 12/5/2026
Sources
Related Entities
Related Claims
UnverifiedIn benchmark testing, Qwen3 27B achieved a generation speed of approximately 40-something tokens per second on 4 modified 2080 Ti cards without NVLink enabled.80% similarUnverified将模型权重从FP16压缩到INT8或INT4的量化技术，可使消费级GPU如RTX 4090在本地部署时达到30-80 TPS的实用速度79% similarVerifiedThe NVIDIA H100 single-card FP64 double-precision floating-point performance reaches approximately 34 TFLOPS.77% similarVerifiedQwen2.5-Coder-32B running with Q4 quantization on an RTX 4090 can achieve an inference speed of 40–60 tokens per second.77% similarUnverifiedAt FP16 half-precision computation, the RTX 4090 has a theoretical performance of approximately 330 TFLOPS, while the M2 Ultra's GPU delivers approximately 27.2 TFLOPS.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/867055API
curl https://kongchang.com/api/v1/knowledge/claims/867055MCP
get_claim(id=867055)