Unverified50% confidenceFactExact time
Token generation speed is divided into two phases: prefill (compute-intensive) and decode (memory-bandwidth-intensive).
1
Sources
50%
Confidence
Long-term
Relevance
10/17/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedMemory bandwidth determines generation speed, because autoregressive decoding moves all weights token by token, and the higher the bandwidth, the more tokens generated per second.78% similarUnverifiedThe Prefill (prompt processing) phase is compute-intensive and relies on the GPU's parallel matrix computation capability; the Decode (generation) phase is memory-bandwidth-intensive and depends more on VRAM bandwidth.77% similarUnverifiedPrefill is a compute-intensive operation where computation grows quadratically with sequence length; decoding is a memory-bandwidth-intensive operation that can reuse the KV Cache.75% similarUnverifiedLong Decode allows output of up to 1M tokens, making it one of the implementations with the largest output window among current frontier models.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1053009API
curl https://kongchang.com/api/v1/knowledge/claims/1053009MCP
get_claim(id=1053009)