Unverified50% confidenceBenchmarkExact time
The 35B model can achieve 45 to 60 tokens per second on a 4090-class GPU.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/12/2026
First Seen
Valid until: 1/10/2027
Sources
Related Entities
Related Claims
VerifiedA 35B model requires approximately 70GB of VRAM at FP16 precision, far exceeding the RTX 5090's 32GB capacity, so single-GPU deployment requires quantization.76% similarUnverifiedA tester ran a 20.7B dense model on a dual-4090 (48 GB VRAM) workstation and achieved 27.6 tokens per second, only slightly faster than Dwarf Star, but with over 10 times fewer parameters.71% similarUnverifiedTaking a 70B BF16 model as an example, generating one token requires reading approximately 140 GB of weights, and when fully utilizing the H100's 3.35 TB/s memory bandwidth, the per-token latency is at least 42 milliseconds, with a theoretical TPS upper limit of approximately 24 tokens/second.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1001696API
curl https://kongchang.com/api/v1/knowledge/claims/1001696MCP
get_claim(id=1001696)