Unverified50% confidenceBenchmarkExact time
With a 5070 12 GB GPU paired with 64 GB of memory, speed can reach 93 tokens/s in short chat scenarios and 74 tokens/s for 128K long contexts.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/5/2026
First Seen
Valid until: 1/3/2027
Sources
Related Entities
Related Claims
UnverifiedA speed of 4–5 tokens/second is too slow for practical use on an 8GB GPU; a 16GB GPU can improve this to 12–14 tokens/second, entering a practically usable range.77% similarUnverifiedOn an RTX 5070 with a 6-core Ryzen and 64 GB of memory, Strata achieves approximately 90 tokens/second for short conversations, approximately 67 tokens/second at a 128,000-token context, and approximately 1,300 tokens/second for reading prompts.77% similarUnverifiedWith an approximately 1.6M ultra-long context and 128 concurrent requests, throughput can reach approximately 3,700 tokens/s (aggregated).75% similarUnverifiedOn a consumer-grade GPU with 8GB of VRAM, ActionAssist achieves an inference speed of 122 tokens/second using Q6_K_L quantization with a 32k context window and 8-bit KV cache quantization.73% similarUnverifiedA UP host running MiniCPM 2B with an RTX 4080S (16GB VRAM) + 64GB RAM configuration achieved an inference speed of approximately 110 tokens/second in real-world testing72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/978781API
curl https://kongchang.com/api/v1/knowledge/claims/978781MCP
get_claim(id=978781)