Neural Memory Module
A class of sub-networks whose parameters can be dynamically updated during model inference, typically implemented as key-value stores or differentiable memory matrices, enabling a model to continually adapt to new data without retraining.
Timeline (last 90 days)
Google's Titans architecture introduces an independent neural memory module that updates its own weights in real time during inference, enabling the model to continuously adapt to new data at the inference stage without retraining the entire model.
A neural memory module is a class of subnetwork whose parameters can be dynamically updated during model inference, usually implemented as key-value storage or a differentiable memory matrix, with its weights themselves adjusted in real time along the input stream.
Titans' contribution lies in combining the neural memory module idea with large-scale language models, letting the memory module play the role of accumulating experience in long-context and streaming-input scenarios.
All Facts (3)
Google's Titans architecture introduces an independent neural memory module that updates its own weights in real time during inference, enabling the model to continuously adapt to new data at the inference stage without retraining the entire model.
50%UnverifiedA neural memory module is a class of subnetwork whose parameters can be dynamically updated during model inference, usually implemented as key-value storage or a differentiable memory matrix, with its weights themselves adjusted in real time along the input stream.
50%UnverifiedTitans' contribution lies in combining the neural memory module idea with large-scale language models, letting the memory module play the role of accumulating experience in long-context and streaming-input scenarios.
50%