Meta-Learning

MAML (Model-Agnostic Meta-Learning) attempts to learn an initialization parameter space that can be quickly fine-tuned with a small number of samples.

The core innovation of MAML lies in finding an initialization parameter point that is highly sensitive to gradient updates, enabling the model to adapt to new tasks with only a small number of gradient steps.

In the face of rapidly iterating model capabilities, establishing a continuous learning mechanism has greater strategic value than one-time deployment.

The recursion at Level 2 is essentially meta-learning, i.e., learning the meta-knowledge of how to build a suitable harness, equivalent to an outer loop.

What makes Reactive Optimizers unique is that it treats the optimizer algorithm itself as the search space, which is closer to the intersection of program synthesis and machine learning.

In the deployed buoy architecture, the meta-learning core is a small time-series CNN with fewer than 50,000 parameters, with a meta-trained initialization and a learned per-layer learning-rate vector.

Adding an anchor task term to the meta-loss, with the anchor weight λ taking a small value of 0.1 to 0.3, yields most of the stability benefit without sacrificing adaptation speed.

Physical RSI requires that the robot not only improve its task capabilities but also improve the "meta-process of improvement capability," and that this meta-learning strategy can transfer across tasks.

The main technical approaches explored for writing memory into weights include continual fine-tuning, meta-learning, and parameter-efficient fine-tuning methods such as LoRA.

The zero-shot capability of the meta-learning approach is essentially limited by the coverage of the system distribution seen during training, while the structural prior approach has stronger extrapolation capability for classes of systems where the prior assumptions hold.

Zero-shot learning is typically achieved via two pathways: meta-learning and generalization based on structural priors.

Algorithm self-discovery is a branch of meta-learning that makes the optimization algorithm itself the object of learning, automatically searching for and generating update rules through interaction with the environment.

Domain generalization requires models to generalize to completely unseen target domain data, with commonly used methods including meta-learning, causal inference-based invariant feature learning, and multi-source domain data augmentation.

Future work may need to explore more abstract target representations, or combine meta-learning strategies so that the agent learns how to extract transferable knowledge from experience.