Continual Learning

Continual learning is a subfield of machine learning that studies how to enable models to continuously acquire new tasks or new data without forgetting previously learned knowledge.

Catastrophic forgetting is a classic problem in the field of neural networks; as early as the 1990s, researchers discovered that the performance on old tasks drops sharply when neural networks are continuously trained on new tasks

Semantic agents with LLMs at their core understand task intent rather than memorizing operation paths, theoretically enabling a significant improvement in cross-version robustness.

An intermittently faulty memory module may alternately produce correct and incorrect computation results, causing sporadic deviations in gradient calculations during neural network training, affecting model convergence without triggering hardware alerts.

Data drift or concept drift refers to the phenomenon where model performance degrades significantly when the real-world data distribution changes while the model is still trained on old data; the industry addresses this with continuous learning and active learning strategies.

Continual training is a method of continuously training on newly added data to keep the model's knowledge up to date.

SFT fine-tuning is effective but causes a decline in general knowledge in small models—i.e., the phenomenon of catastrophic forgetting—due to the prominent capacity bottleneck of small models.

Continual Learning is the ability of AI to continuously learn from new experiences without forgetting existing knowledge; current large language models are limited by static weights that are frozen after training.

The Sleep framework proposes a new path for LLMs to retain existing knowledge during continual learning, and its core idea draws on the mechanism of memory consolidation during biological sleep.

For code review scenarios, possible data sources for continuous learning include commit history, code change patterns, PR discussion records, and developer feedback on comments.

In software engineering, continuous learning usually refers to a system dynamically updating models or rules based on new data without requiring full retraining.

The median improvement of 5.3 percentage points indicates that current models are mostly doing 'local patching' and have not yet formed cross-level knowledge accumulation, which is the key difference between continual learning and single-round self-correction.

RCSD's training framework has continual learning characteristics and can adapt to domain-specific data distributions.

Traditional continual learning in the exemplar-free setting is prone to catastrophic forgetting, overwriting prior knowledge.

The paper formulates the OOD detection problem in continual learning as two constraints: (i) learning a sequence of tasks without catastrophic forgetting; (ii) identifying OOD samples without access to past task ID data.

In continual learning, standard confidence scores shift systematically as tasks progress: the model is overconfident on inputs from recent tasks and gives muddled confidence for old tasks and truly unknown inputs.

TA-KAN is a framework for rehearsal-free continual learning scenarios, aiming to simultaneously reduce forgetting of old tasks and accurately identify out-of-distribution (OOD) samples.

Gating (energy gate, novelty gate, safety gate) reduced average hourly energy consumption by about 60% compared with per-sample updates, with no measurable loss in adaptation quality.

After switching to frontier LLMs, the team gained two additional benefits, in-context learning and continual learning: the robot can quickly understand new tasks through a small number of examples without retraining; when encountering new situations, the large model intervenes to adjust the entire pipeline, enabling the system to continuously evolve over deployment time.

To achieve continuous growth, the problem of 'catastrophic forgetting' must be solved - training on new data overwrites old knowledge.