Unverified50% confidenceBenchmarkExact time
In the AgentX test, vLLM's P90 per-user throughput was 196 tokens/s.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/29/2026
First Seen
Valid until: 11/12/2026
Sources
Related Entities
Related Claims
UnverifiedAccording to the FreeToken technical paper, Qwen3-235B-A3B can achieve 77–83 tokens/s on an RTX 5090 system, and DeepSeek-related models can achieve 22–25 tokens/s on agent trajectories.65% similarUnverifiedIn a llama.cpp environment with Qwen3 27B, the R9700 achieved 27.1 tokens/s (approximately 65% of bandwidth ceiling) and the B70 achieved 21.4 tokens/s (just over approximately 50%).64% similarUnverifiedA single RTX 5090 with LLaDA-2.1-mini + ORL reaches 2,788 tokens/s at NVFP4 precision (single-request decoding, batch size 1).62% similarUnverifiedIn single-user scenarios llama.cpp runs an 8B model at about 115 tok/s with about 12ms first-token latency on an RTX 4090, but at 32 concurrency throughput is only about 210 tok/s.60% similarUnverifiedOn an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1110156API
curl https://kongchang.com/api/v1/knowledge/claims/1110156MCP
get_claim(id=1110156)