Unverified70% confidenceFactExact time
LLM inference during the decode phase is typically memory-bandwidth-bound rather than compute-bound, because token-by-token generation requires loading the entire model weights but has extremely low arithmetic intensity.
2
Sources
70%
Confidence
Long-term
Relevance
9/10/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedLLM 推理分为预填充阶段（Prefill，计算密集型）和解码阶段（Decode，内存带宽密集型）两个阶段80% similarUnverifiedLLM inference is autoregressive token-by-token generation with bottlenecks in memory bandwidth and KV Cache management, which is fundamentally different from diffusion model inference.80% similarUnverifiedThe decode phase of large language model inference generates only one token at a time and is a memory-bandwidth-bound operation, because generating each token requires loading the entire model weights from memory once.79% similarUnverifiedThe autoregressive decoding phase of LLMs is limited by memory bandwidth bottlenecks, as generating each token requires a complete traversal of the activated model weights.78% similarUnverifiedLLM inference costs are primarily composed of two parts: the Prefill stage for processing input tokens and the Decode stage for generating output tokens, with Prefill accounting for the vast majority of overhead in code understanding scenarios.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/890066API
curl https://kongchang.com/api/v1/knowledge/claims/890066MCP
get_claim(id=890066)