Unverified60% confidenceFactExact time
The autoregressive generation mechanism of the Transformer architecture means that input tokens only require a single forward-pass encoding and are stored and reused via KV Cache, whereas each step of output token generation requires a full forward pass, causing inference cost to grow linearly with output length.
2
Sources
60%
Confidence
Long-term
Relevance
7/18/2026
First Seen
Sources
Related Claims
UnverifiedTransformer 架构中自注意力层会为每个 Token 生成对应的 Key 和 Value 向量并缓存在 GPU 显存中，即 KV Cache 机制，上下文越长占用显存越大，推理成本呈近线性增长78% similarUnverifiedPrompt caching relies on the Transformer's KV cache mechanism, skipping redundant computation by reusing already-computed KV vectors for identical prefixes, saving GPU compute and reducing time-to-first-token latency.76% similarUnverifiedThe Looped Transformer repeatedly calls the same set of layers in a loop, thereby inserting far more serial computation steps between two text outputs than in a conventional forward pass.75% similarVerifiedBy reusing the same weight matrices, the Recurrent Transformer can achieve deeper levels of information processing without significantly increasing the number of parameters.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/554778API
curl https://kongchang.com/api/v1/knowledge/claims/554778MCP
get_claim(id=554778)