Continual-learning rules shape representational drift
Abstract
Lifelong learning requires acquiring new knowledge without erasing the old. Yet neural population codes for familiar stimuli and behaviors change over days and weeks. This coexistence of stable memory and changing internal codes may depend on how a learning system prevents forgetting. We therefore tested whether different continual-learning mechanisms produce distinct patterns of representational drift. We trained convolutional networks on sequential image classification tasks and recurrent networks on sequences of cognitive tasks, tracking fixed probe representations across learning. Experience replay preserved earlier tasks in both architectures while representations drifted progressively with the number of intervening tasks. Drift was structured: later visual-processing stages and recurrent units' temporal tuning were especially labile, whereas coarse class organization and task-relevant temporal structure persisted. In contrast, algorithm that strongly anchored weights nearly froze representations. Directly anchoring an old representation during replay likewise suppressed drift and impaired acquisition of subsequent tasks. Together, these results link representational drift to the stability--plasticity trade-off: its magnitude is shaped by the mechanism that protects old knowledge, and suppressing it can restrict future learning. Drift may therefore provide an observable signature of the constraints that enable continual learning in brains and machines.
Related
- Discrete synaptic states and context-modulated readouts support continual learning
- Multidimensional semantic representations emerge from multi-frequency representational similarity learning
- Representational learning by optimization of neural manifolds in an olfactory memory network
- Representational geometry of punishment learning across basolateral amygdala neurons
- Constrained Hebbian learning supports efficient representational allocation under structural constraints.
BibTeX
@misc{si2026continuallearning,
title = {{Continual-learning rules shape representational drift}},
author = {Yikai Si and Shanshan Qin},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.16141},
eprint = {2608.16141},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.16141},
}
RIS
TY - UNPB
TI - Continual-learning rules shape representational drift
AU - Yikai Si
AU - Shanshan Qin
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2608.16141
UR - https://arxiv.org/abs/2608.16141
AB - Lifelong learning requires acquiring new knowledge without erasing the old. Yet neural population codes for familiar stimuli and behaviors change over days and weeks. This coexistence of stable memory and changing internal codes may depend on how a learning system prevents forgetting. We therefore tested whether different continual-learning mechanisms produce distinct patterns of representational drift. We trained convolutional networks on sequential image classification tasks and recurrent networks on sequences of cognitive tasks, tracking fixed probe representations across learning. Experience replay preserved earlier tasks in both architectures while representations drifted progressively with the number of intervening tasks. Drift was structured: later visual-processing stages and recurrent units' temporal tuning were especially labile, whereas coarse class organization and task-relevant temporal structure persisted. In contrast, algorithm that strongly anchored weights nearly froze representations. Directly anchoring an old representation during replay likewise suppressed drift and impaired acquisition of subsequent tasks. Together, these results link representational drift to the stability--plasticity trade-off: its magnitude is shaped by the mechanism that protects old knowledge, and suppressing it can restrict future learning. Drift may therefore provide an observable signature of the constraints that enable continual learning in brains and machines.
ER -