Unverified50% confidenceFactExact time
Hybrid linear attention (HLA) models retain standard attention in some layers and use linear attention in the rest, balancing precise local retrieval with long-range memory.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe model's layer composition is 54 layers of KDA (Kimi Delta Attention) linear attention, 17 layers of BCSA compressed sparse attention, and 1 full attention layer (layer 72).77% similarUnverifiedHybrid architectures (linear attention + standard attention) are more efficient than pure full attention on long-context tasks, but linear attention may be slightly inferior to standard attention on certain reasoning tasks that require precise global dependencies76% similarUnverifiedCommon attention mechanism modules for model improvement include the SE attention mechanism, CBAM channel and spatial attention, and ECA efficient channel attention.75% similarUnverifiedGLM-5.3-Flash 的注意力架构将计算拆为线性注意力（O(n)递归状态，处理近期token）和稀疏注意力（轻量索引器回扫全局上下文，只拉取相关远端片段）两条路径。74% similarUnverifiedExisting mainstream sparse attention methods filter interactions by retaining entries with larger values or high-quality regions in the attention matrix.74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1032831API
curl https://kongchang.com/api/v1/knowledge/claims/1032831MCP
get_claim(id=1032831)