Unverified50% confidenceBenchmarkExact time
The original configuration's prefill speed was below 200 tokens/second, partly because the micro-batch size was set to 128; after raising the micro-batch to 1024 and moving some expert layers back to the GPU, prefill speed increased to about 800 tokens/second.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/29/2026
First Seen
Valid until: 1/27/2027
Sources
Related Entities
Related Claims
UnverifiedUnder this configuration, the 35B model achieves a measured single-user peak speed of approximately 80 tokens/second, with concurrent throughput exceeding 500 tokens/second.73% similarUnverifiedWhen running Qwen3-Next-Flash (Autoround W4A16 quantization), the prefill speed was approximately 1.3k tokens/s, and the generation speed reached 70 tokens/s (code) / 60 tokens/s (prose), with support for 128k+ long context.71% similarUnverifiedQuantprobe claims to enable a machine with only 6GB of VRAM and 16GB of RAM to run 30B-class models at 22 tokens/s.71% similarUnverifiedWhen running 4-bit DeepSeek R1 on the M3 Ultra, a 1,000-token context generates approximately 19 tokens per second using 392GB, dropping to approximately 15 tokens per second using 410GB at a 16,000-token context.71% similarUnverifiedRunning the model on Windows is approximately 20% slower than on Linux, with approximately 16 tokens/s for short contexts and approximately 10 tokens/s for long contexts.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1112086API
curl https://kongchang.com/api/v1/knowledge/claims/1112086MCP
get_claim(id=1112086)