Unverified50% confidenceBenchmarkExact time
对于BERT-base模型推理会话初始化通常耗时数百毫秒，重复执行会使延迟增加1-2个数量级
1
Sources
50%
Confidence
Long-term
Relevance
7/7/2026
First Seen
Sources
14倍提速：Manticore如何重构ONNX嵌入推理路径
hackernewshackernews7/3/2026
Related Claims
VerifiedA single LLM inference call can easily take hundreds of milliseconds or even seconds, whereas the execution time of deterministic code is typically at the microsecond level.76% similarUnverifiedIn orchestrator scenarios, the number of model calls can be 3 to 10 times that of a single-turn conversation.71% similarUnverifiedThe gemma:2b model may require a response time of several seconds to over ten seconds for local inference on an ordinary machine.71% similarUnverified流式推理将输入切分为时间帧（通常20-80毫秒一帧），模型对持续到来的每一帧持续更新内部状态并输出增量结果71% similarUnverifiedIn actual testing, a single classification by the Dify question classifier takes approximately 11 seconds, because execution speed depends on the large model's response.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/176728API
curl https://kongchang.com/api/v1/knowledge/claims/176728MCP
get_claim(id=176728)