Unverified50% confidenceFactExact time
TTFT is mainly determined by the prefill phase, which is the compute bottleneck and depends on GPU matrix multiplication throughput; decode speed is memory-bandwidth-bound, requiring model weights and KV cache to be moved from memory to the compute units.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
Unverified长上下文在首次推理时需完整计算KV Cache，消耗GPU计算资源并占用显存带宽，拉长首token延迟（TTFT）77% similarUnverifiedThe prefill stage is a compute-intensive operation primarily limited by GPU compute, while the decode stage is a memory-bandwidth-intensive operation primarily limited by VRAM bandwidth.73% similarUnverified在多节点 prefill/decode 分离架构中，prefill 节点需将生成的 KV cache 通过高速网络传给 decode 节点，节点间通信是关键瓶颈，KV cache 传输效率直接决定分离架构能否兑现性能收益。72% similarVerifiedFor modern CPUs and GPUs, memory bandwidth is often the true bottleneck for inference speed rather than the compute units' arithmetic throughput, which is why quantization can bring speed improvements.72% similarUnverifiedGeneration throughput (tok/s) reflects decoding-phase efficiency and is affected by KV Cache size and memory bandwidth, with far lower sensitivity to input length than prefilling.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1038334API
curl https://kongchang.com/api/v1/knowledge/claims/1038334MCP
get_claim(id=1038334)