Unverified50% confidenceBenchmarkExact time
In tests with the 4-bit quantized version (fitting entirely into 32 GB of VRAM), Ollama reached 240 tokens/s while FreeToken reached 225 tokens/s, making Ollama actually faster.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/11/2026
First Seen
Valid until: 12/10/2026
Sources
Related Entities
Related Claims
UnverifiedWhen running at 1-bit precision on an M2 Max equipped with 96GB of VRAM, generation speed for simple tasks is consistently around 28–30 tokens/second69% similarUnverifiedWhen VRAM is sufficient and the model is fully resident on the GPU, an RTX 4090 running a Q4-quantized 27B model can typically achieve 15–30 tokens/s; once CPU offloading is triggered, the speed may drop sharply to 1–2 tokens/s or even lower.68% similarUnverifiedA tester ran a 20.7B dense model on a dual-4090 (48 GB VRAM) workstation and achieved 27.6 tokens per second, only slightly faster than Dwarf Star, but with over 10 times fewer parameters.67% similarUnverifiedOn a single 32GB GPU under q4_0 quantization, the context length increased from 82,432 tokens before the update to 199,680 tokens after the update.67% similarUnverifiedCommunity benchmark comparison data: M2 Ultra approximately 20 tokens/second, M3 Ultra 512 GB at 39.8 tokens/second, a single B200 at 119.7 tokens/second, 8× RTX Pro 6000 at approximately 204 tokens/second, a modified RTX 4090 with 48 GB VRAM on a custom backend at 367 tokens/second, and dual H200 at approximately 375 tokens/second.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/897263API
curl https://kongchang.com/api/v1/knowledge/claims/897263MCP
get_claim(id=897263)