S4
A Structured State Space Sequence model and an early representative SSM architecture. It applies special structure to matrix A via HiPPO initialization to capture long-range dependencies efficiently, and serves as the foundation for subsequent works such as Mamba.
Timeline (last 90 days)
The experiments focus on discrete groups such as S4 and sparse graphs; extension to larger scales, continuous groups, and dense graphs has not yet been carried out.
Most SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.
Typical representatives of state space models include S4, Mamba and others, which write the recurrence as a continuous differential equation or discrete linear recurrence and achieve efficient parallel training with the help of convolution or selective scan.
Early SSMs such as S4 efficiently capture long-range dependencies by applying HiPPO initialization to matrix A, while maintaining the inference efficiency of linear recurrence.
Gu等人在2021年的HiPPO和S4工作中将状态空间模型引入深度学习，开创了结构化状态空间模型研究方向
All Facts (5)
The experiments focus on discrete groups such as S4 and sparse graphs; extension to larger scales, continuous groups, and dense graphs has not yet been carried out.
50%UnverifiedMost SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.
50%UnverifiedTypical representatives of state space models include S4, Mamba and others, which write the recurrence as a continuous differential equation or discrete linear recurrence and achieve efficient parallel training with the help of convolution or selective scan.
50%UnverifiedEarly SSMs such as S4 efficiently capture long-range dependencies by applying HiPPO initialization to matrix A, while maintaining the inference efficiency of linear recurrence.
50%UnverifiedGu等人在2021年的HiPPO和S4工作中将状态空间模型引入深度学习，开创了结构化状态空间模型研究方向
50%