GDN

A recurrent state model architecture based on linear attention. It introduces a gating mechanism to control the forgetting rate of historical information and uses the Delta rule to make state updates approximate online gradient steps.