Deep Equilibrium Models
A sequence model architecture that replaces stacked layers with an implicitly solved equilibrium point. By solving for a fixed point, it achieves the equivalent of infinite depth and is one of the theoretical sources of recurrent and weight-sharing architectures.
Timeline (last 90 days)
Implicit depth models such as Deep Equilibrium Models have a theoretical resonance with MetaPred's approach of using convergence as the training objective.
The idea of recurrent architecture can be traced back to the Universal Transformer and Deep Equilibrium Models, boosting reasoning ability by increasing computational depth without increasing the number of parameters.
All Facts (2)
Implicit depth models such as Deep Equilibrium Models have a theoretical resonance with MetaPred's approach of using convergence as the training objective.
50%UnverifiedThe idea of recurrent architecture can be traced back to the Universal Transformer and Deep Equilibrium Models, boosting reasoning ability by increasing computational depth without increasing the number of parameters.
50%