Evolutionary Approaches to Continual Machine Learning

Summary & Methodology Analysis

Traditional continual learning relies on offline parameter updates, which is increasingly insufficient for modern agentic systems and large language models. The authors propose a taxonomy based on three axes: When, where the capability resides, and how the system updates. The When axis covers pre-training, post-training, and inference-time adaptation. The Where axis expands the focus from weights to external components like memory, skills, and protocols. The How axis transitions from traditional off-policy gradient updates to on-policy reinforcement learning and beyond-gradient optimization. This structure maps existing approaches, such as TTA, TTT, and various memory-augmented systems, into a unified space to highlight research gaps. 1 The experimental analysis confirms that on-policy strategies effectively mitigate catastrophic forgetting, a common failure mode where new data overwrites previous knowledge. Furthermore, the paper highlights zeroth-order optimization methods like MeZO as a stable alternative for fine-tuning, as they operate solely on forward evaluations. This is particularly relevant for systems where gradients are unavailable or prohibitively expensive to compute during deployment. 1 Despite the utility of this taxonomy, the paper acknowledges that the boundaries between these three axes can overlap, leading to potential classification ambiguity. Evaluation protocols for complex, cross-axis methods remain in an early stage of development, and the current study does not account for preprints released after the submission deadline. The authors note that standardizing performance metrics for these evolving paradigms remains a significant challenge for the field.