Expired50% confidenceBenchmarkExact time
A 3B model typically achieves an inference speed of 10–30 tokens/second on a modern CPU via llama.cpp.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/16/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/14/2026(expired)
Sources
Related Claims
UnverifiedThrough pure CPU inference and the 4-bit GGUF quantization format, llama.cpp enables a 7B-parameter model to achieve a generation speed of 20–40 tokens per second on an ordinary server without a GPU.85% similarUnverifiedIn the STM32 embedded development test, the locally deployed DS4 model had an average output speed of approximately 23 tokens/second, memory usage spiked to 110 GB during inference, and the peak speed reached 83.8 tokens/second.74% similarPartially VerifiedFor the 4-bit quantized version of a 7B parameter model, at least 8GB of memory and a modern multi-core CPU are typically required for smooth operation, with inference speeds of approximately 10–30 tokens per second73% similarUnverifiedBased on the MLX framework, top-configured Mac models can achieve a generation speed of approximately 20 tokens/second when running the Llama-3-70B quantized model.70% similarUnverifiedUnder a 4×RTX 3090 SXM4 configuration, the quantized version of GLM-5.2 achieves a token generation speed of approximately 6.3–6.8 tokens/second and a prompt evaluation speed of approximately 10.6 tokens/second.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/532785API
curl https://kongchang.com/api/v1/knowledge/claims/532785MCP
get_claim(id=532785)