Discrete synaptic states and context-modulated readouts support continual learning
Abstract
Biological neural systems learn new behaviors while retaining earlier ones, whereas sequentially trained artificial networks often overwrite parameters and forget previously learned behaviors. We introduce the Context-modulated Synaptic-states Recurrent Neural Network (CoSyn-RNN), a new continual-learning model inspired by heterogeneous synaptic states and neuromodulatory control. CoSyn-RNN combines sparse recurrent allocation with a discrete protection rule: after each task, recurrent neurons whose synaptic weights cross a fixed threshold form a task-specific neuron group, and their associated parameters enter a protected state during later learning. A neuron-specific gain and a task-cued mask over a fixed base readout control how recurrent activity contributes to behavior. Across various task sequences consisting of many cognitive tasks, training and validation losses and accuracies were maintained throughout learning. The protection rule produced modular, task-ordered recurrent structure compatible with forward reuse of earlier computations. Pairwise recruitment measurements further defined minimum- and maximum-cost curricula that produced markedly different final recurrent structures and different remaining capacity for future learning. Thus, CoSyn-RNN provides a biologically inspired model in which protected synaptic states, sparse allocation, and context-dependent modulation support continual learning.
Related
- Contact-Free Nuclear and Cytoplasmic Mechanophenotyping by Amplitude-Modulated Ultrasound and Phase-Locked Optical Diffraction Tomography: Computational Feasibility and Initial Experimental Readouts
- Continual-learning rules shape representational drift
- A synaptic locus of song learning
- Synaptic proteomics identifies cathepsin B as a regulator of synapse remodelling during learning
- Ethological learning during the critical period resets synaptic setpoints in mouse binocular visual cortex
BibTeX
@misc{gao2026discrete,
title = {{Discrete synaptic states and context-modulated readouts support continual learning}},
author = {Yuan Gao and Stefan Mihalas and Denis Turcu},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.11.750704},
url = {https://doi.org/10.64898/2026.09.11.750704},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Discrete synaptic states and context-modulated readouts support continual learning
AU - Yuan Gao
AU - Stefan Mihalas
AU - Denis Turcu
PY - 2026
DA - 2026/09/11
PB - bioRxiv
DO - 10.64898/2026.09.11.750704
UR - https://doi.org/10.64898/2026.09.11.750704
AB - Biological neural systems learn new behaviors while retaining earlier ones, whereas sequentially trained artificial networks often overwrite parameters and forget previously learned behaviors. We introduce the Context-modulated Synaptic-states Recurrent Neural Network (CoSyn-RNN), a new continual-learning model inspired by heterogeneous synaptic states and neuromodulatory control. CoSyn-RNN combines sparse recurrent allocation with a discrete protection rule: after each task, recurrent neurons whose synaptic weights cross a fixed threshold form a task-specific neuron group, and their associated parameters enter a protected state during later learning. A neuron-specific gain and a task-cued mask over a fixed base readout control how recurrent activity contributes to behavior. Across various task sequences consisting of many cognitive tasks, training and validation losses and accuracies were maintained throughout learning. The protection rule produced modular, task-ordered recurrent structure compatible with forward reuse of earlier computations. Pairwise recruitment measurements further defined minimum- and maximum-cost curricula that produced markedly different final recurrent structures and different remaining capacity for future learning. Thus, CoSyn-RNN provides a biologically inspired model in which protected synaptic states, sparse allocation, and context-dependent modulation support continual learning.
SN - 2692-8205
ER -