Unverified50% confidenceBenchmarkExact time
On an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/16/2026
First Seen
Valid until: 12/15/2026
Sources
Related Entities
Related Claims
UnverifiedA UP host running MiniCPM 2B with an RTX 4080S (16GB VRAM) + 64GB RAM configuration achieved an inference speed of approximately 110 tokens/second in real-world testing76% similarUnverifiedThe model can achieve approximately 206 tokens/second on a single RTX 5090 using the NVFP4 scheme.74% similarUnverifiedCommunity benchmark comparison data: M2 Ultra approximately 20 tokens/second, M3 Ultra 512 GB at 39.8 tokens/second, a single B200 at 119.7 tokens/second, 8× RTX Pro 6000 at approximately 204 tokens/second, a modified RTX 4090 with 48 GB VRAM on a custom backend at 367 tokens/second, and dual H200 at approximately 375 tokens/second.70% similarUnverifiedThe RTX 4090 has a theoretical FP16 compute performance of 330 TFLOPS, and the actual Decode speed for a 7B FP16 model is typically 50–60 tokens/s.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/924474API
curl https://kongchang.com/api/v1/knowledge/claims/924474MCP
get_claim(id=924474)