Unverified50% confidenceFactExact time
自回归解码在批大小为1时算术强度约等于隐藏层维度（通常4096至16384），而现代GPU峰值算术强度往往超过200，导致解码处于内存带宽受限状态
1
Sources
50%
Confidence
Long-term
Relevance
7/8/2026
First Seen
Sources
Related Claims
VerifiedTaking a 70B parameter model with FP16 precision and 32 attention head layers as an example, the KV Cache for a single 200K-length request requires approximately 80–120 GB of VRAM, far exceeding the 80 GB VRAM limit of a single A100 GPU.72% similarVerifiedA GPT-4-class model with 175 billion parameters requires approximately 175 GB of VRAM even when stored at fp8 precision.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/268031API
curl https://kongchang.com/api/v1/knowledge/claims/268031MCP
get_claim(id=268031)