Expired50% confidenceBenchmarkExact time
In the same test, the bf16 baseline exceeded the self-developed W4 kernel (25.5 tok/s) and bitsandbytes NF4 (25.7 tok/s) at 31.3 tok/s, reaching 29.8% of the theoretical roofline.
1
Sources
50%
Confidence
Short-term (~14 days)
Relevance
10/12/2026
First Seen
This claim has expired. It is kept for the record and no longer describes the current state.
Valid until: 10/26/2026(expired)
Sources
Related Entities
Related Claims
UnverifiedFor the Qwen2.5-1.5B model, the int4 theoretical roofline ceiling for batch-1 decoding is 411.8 tok/s, while the actual self-developed kernel reaches only 25.5 tok/s, a gap of about 16x.73% similarUnverifiedThe self-developed W4 kernel's perplexity (probe ppl) is 15.76, better than bitsandbytes NF4's 18.98.73% similarUnverifiedBF16 uses 1 sign bit, 8 exponent bits, and 7 mantissa bits, sacrificing precision while retaining the same dynamic range as FP32.69% similarUnverifiedThe performance gap between a carefully quantized 4-bit model and the original FP16 model is typically within 1–3 percentage points.69% similarUnverified在27B混合架构、4B稠密模型、36B MoE三类模型上以8k上下文测试，Prefill阶段性能无变化，Decode阶段打补丁后快约1.4%68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1006834API
curl https://kongchang.com/api/v1/knowledge/claims/1006834MCP
get_claim(id=1006834)