Unverified50% confidenceFactExact time
The paper evaluated three configurations with 10.94M parameters each on the TinyStories dataset: pure Transformer, pure wave, and hybrid wave-attention, with context lengths up to 4096 tokens.
1
Sources
50%
Confidence
Long-term
Relevance
10/30/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTransformer推理中批量大小为1时注意力层的算术强度约1~10 FLOP/Byte，远低于H100约300 FLOP/Byte的屋脊点，属于带宽受限场景70% similarUnverified标准Transformer自注意力计算量随序列长度平方增长，50万token的注意力计算量约为4096token的15000倍64% similarUnverifiedTransformer注意力机制的计算量随序列长度平方增长，早期语言模型上下文窗口上限约4000个token64% similarUnverifiedTransformer注意力机制中的矩阵乘法和Softmax运算占据推理总算力的70%以上，ASIC可实现3至10倍的能效提升64% similarUnverifiedTiny-MoE模型总参数量约200M，但每个token激活的参数仅约90M63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1115528API
curl https://kongchang.com/api/v1/knowledge/claims/1115528MCP
get_claim(id=1115528)