Unverified50% confidenceBenchmarkExact time
On an 8GB GPU, the token generation speed is approximately 4.2 to 5 tokens/second, and decreases from around 6 tokens/second to approximately 4.2 as the context grows longer.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
VerifiedHuman reading speed is roughly 5-8 tokens per second73% similarVerifiedIn decode speed tests, speeds of approximately 45 tokens/second are achievable within 20,000 tokens, dropping sharply to approximately 20 tokens/second beyond 30,000 tokens.72% similarUnverifiedLocal model inference speed is approximately 1–10 tokens/second, while cloud-based APIs can reach 50–150 tokens/second.71% similarUnverifiedIn this test, the model's output speed in short-context scenarios is approximately 20 tokens/second, dropping to approximately 13 tokens/second for contexts of 200K+.71% similarUnverifiedWhen Tidy processes short text (100–500 tokens), inference latency is approximately 1–2 seconds for initial loading plus 200–800 milliseconds of computation, with continuous use impacting battery life by less than 5% per hour.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/907302API
curl https://kongchang.com/api/v1/knowledge/claims/907302MCP
get_claim(id=907302)