Unverified50% confidenceBenchmarkExact time
When running the Llama 3.1 8B model on an i7-100, SGLang reaches about 16200 tokens/s and vLLM about 12500 tokens/s, a difference of about 30%.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/28/2026
First Seen
Valid until: 11/11/2026
Sources
Related Entities
Related Claims
UnverifiedUnder high power operation, the 35B MoE model reaches 78 to 107 tokens/s, while the dense 27B model only reaches 22 tokens/s.76% similarUnverifiedThe baseline BF16 model reaches about 37.5 tokens/s at single concurrency, 714 tokens/s aggregate throughput at 32 concurrent requests, and 61ms first-token latency; after switching to FP8 weights, single concurrency rises to 53.4 tokens/s, and peak throughput at 64 concurrency rises from about 800 to 1,140 tokens/s, about 1.42x.74% similarUnverifiedOn the 120-billion-parameter GPT-OSS 120B, token generation improves from 90 to 130 tokens/s, and PP improves from approximately 1,300 to approximately 3,200 tokens/s.72% similarUnverifiedIn a text generation test using an 8-billion-parameter model, the M5 Ultra achieved 188 tokens per second versus 129 tokens per second for the M3 Ultra, representing only 1.45x.71% similarUnverifiedWhen running Qwen3-Next-Flash (Autoround W4A16 quantization), the prefill speed was approximately 1.3k tokens/s, and the generation speed reached 70 tokens/s (code) / 60 tokens/s (prose), with support for 128k+ long context.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1106443API
curl https://kongchang.com/api/v1/knowledge/claims/1106443MCP
get_claim(id=1106443)