Unverified50% confidencePredictionExact time
The UP host estimates that inference speed could improve to 30–40 tokens/s after using a GPU with 50GB of VRAM (not confirmed through actual testing).
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
UnverifiedUnder this configuration, the 35B model achieves a measured single-user peak speed of approximately 80 tokens/second, with concurrent throughput exceeding 500 tokens/second.74% similarUnverifiedIn the STM32 embedded development test, the locally deployed DS4 model had an average output speed of approximately 23 tokens/second, memory usage spiked to 110 GB during inference, and the peak speed reached 83.8 tokens/second.70% similarUnverifiedIn prefill speed tests, Halo achieved approximately 650 tokens/s while Spark achieved approximately 2000 tokens/s, with Spark being nearly three times faster.70% similarUnverifiedThe author considers 12–30 tokens/s to be the comfort zone, with above 30 tokens/s being good; an M5 Max with 48–64 GB can run at approximately 20 tokens/s.68% similarUnverifiedA 200K token context request requires approximately 25× the KV-Cache memory compared to an 8K token request.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/907339API
curl https://kongchang.com/api/v1/knowledge/claims/907339MCP
get_claim(id=907339)