Unverified50% confidenceFactExact time
流式推理将输入切分为时间帧（通常20-80毫秒一帧），模型对持续到来的每一帧持续更新内部状态并输出增量结果
1
Sources
50%
Confidence
Long-term
Relevance
7/20/2026
First Seen
Sources
Related Claims
Unverified对于BERT-base模型推理会话初始化通常耗时数百毫秒，重复执行会使延迟增加1-2个数量级71% similarVerifiedA single LLM inference call can easily take hundreds of milliseconds or even seconds, whereas the execution time of deterministic code is typically at the microsecond level.71% similarUnverified对于内容审核这种需要实时检查每次交互的场景，3B模型的单次推理通常可控制在50-100毫秒以内，而百亿级模型可能需要数百毫秒70% similarUnverified代码补全场景要求模型在用户键入的几百毫秒内返回补全建议，需要Fill-in-the-Middle（FIM）能力69% similarVerifiedThe response time tolerance for code completion in AI programming tools is typically within 200–500 milliseconds.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/568266API
curl https://kongchang.com/api/v1/knowledge/claims/568266MCP
get_claim(id=568266)