Unverified50% confidenceBenchmarkExact time
单条推理通常只能占用GPU SM利用率的5%～15%，而合理的批次大小可将利用率提升至60%～90%
1
Sources
50%
Confidence
Long-term
Relevance
7/7/2026
First Seen
Sources
14倍提速：Manticore如何重构ONNX嵌入推理路径
hackernewshackernews7/3/2026
Related Claims
Unverified许多推理服务的GPU利用率不足30%，通过VGPU技术可将利用率提升至70%以上80% similarUnverifiedDuring large model inference, actual GPU utilization is often less than 10%.76% similarUnverifiedGPU utilization can rise from 20%–30% during low traffic to 70%–80% during high traffic, because larger request volumes make it easier to aggregate into larger compute batches75% similarUnverifiedIn a complete AI application system, GPU inference typically accounts for only 30%–40% of total compute, with the remaining 60%–70% of the compute load borne by the CPU.75% similarUnverifiedDuring testing, only 1 of the 4 GPUs reached a utilization rate of approximately 10%–13%, while the rest were nearly idle, and CPU usage was as high as approximately 50%.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/176723API
curl https://kongchang.com/api/v1/knowledge/claims/176723MCP
get_claim(id=176723)