Unverified50% confidenceBenchmarkExact time
In the AgentX test, vLLM's total throughput reached 954K tokens/s/GPU (including cached, input, and output tokens).
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/29/2026
First Seen
Valid until: 11/12/2026
Sources
Related Entities
Related Claims
UnverifiedSGLang using official NVFP4 weights with D-Spark speculative decoding achieved an average of approximately 173 tokens/second on Blackwell GPUs, approaching 200 for short tasks, while loading a 262k context window.70% similarUnverifiedReal-world testing with the 284-billion-parameter DeepSeek V4 Flash shows token generation rising from 37 to 53 tokens/s (approximately 1.5x), and Prompt Processing jumping from 483 to 1,485 tokens/s (approximately three times).69% similarUnverifiedOn an A100 (80 GB VRAM) GPU, the throughput for training a 7B model is approximately 3,000–5,000 tokens per second.68% similarUnverifiedThe output token unit price of large model APIs is typically 2–4 times that of input tokens, because the autoregressive generation mechanism requires one independent forward pass per output token and KV Cache occupies a large amount of GPU memory.68% similarUnverifiedvLLM's PagedAttention splits the KV cache into fixed-size physical blocks (typically 16 tokens), improving GPU memory utilization from below 40% to close to 90%.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1110155API
curl https://kongchang.com/api/v1/knowledge/claims/1110155MCP
get_claim(id=1110155)