Unverified50% confidenceFactExact time
The computational complexity of the standard self-attention mechanism is proportional to the square of the context length (O(n²)), meaning extending the context from 100K to 1M theoretically increases computation by 100 times.
1
Sources
50%
Confidence
Long-term
Relevance
8/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe computational load of the attention mechanism is proportional to the square of the sequence length, meaning doubling the context increases computation fourfold.78% similarUnverified蒙特卡洛方法的收敛速度为O(1/√N)，精度每提高一个数量级需要样本量增加百倍74% similarUnverified选择性重计算由Korthikanti等人在2022年的Megatron-LM论文中系统研究，仅对注意力激活值做重计算可在额外计算开销不超过10%的条件下接近全量重计算的显存节省69% similarUnverifiedTransformer注意力机制的计算量随序列长度平方增长，早期语言模型上下文窗口上限约4000个token66% similarUnverifiedTransformer注意力机制中的矩阵乘法和Softmax运算占据推理总算力的70%以上，ASIC可实现3至10倍的能效提升65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/822578API
curl https://kongchang.com/api/v1/knowledge/claims/822578MCP
get_claim(id=822578)