Stabilization of recurrent neural networks through divisive normalization
5 affiliations
- 1Center for Neural Science, New York University
- 2Department of Physics, Center for Soft Matter Research, New York University
- 3Courant Institute of Mathematical Sciences, New York University
- 4Department of Psychology, New York University
- 5Department of Chemistry, Simons Center for Computational Physical Chemistry, New York University
Abstract
Stability is a fundamental requirement for both biological and engineered neural circuits, yet it is surprisingly difficult to guarantee in the presence of recurrent interactions. Standard linear dynamical models of recurrent networks are sensitive to the precise values of the synaptic weights, since stability requires all eigenvalues of the recurrent matrix to lie within the unit circle. Here we demonstrate, both theoretically and numerically, that an arbitrary recurrent neural network can remain stable even when its spectral radius exceeds 1, provided it incorporates divisive normalization, a dynamical neural operation that suppresses the responses of individual neurons. Sufficiently strong recurrent weights lead to instability, but the approach to the unstable phase is preceded by a regime of critical slowing down, a well-known early warning signal for loss of stability. Remarkably, the onset of critical slowing down coincides with the breakdown of normalization, which we predict analytically as a function of the synaptic strength and the magnitude of the external input. Our findings suggest that the widespread implementation of normalization across neural systems may derive not only from its computational role, but also to enhance dynamical stability. I. SIGNIFICANCE STATEMENT Neural circuits must remain stable to function correctly, yet strong recurrent connections, which are essential for complex computations, often hinder this stability. We demonstrate that divisive normalization, a canonical neural operation found across many sensory systems, can actively stabilize neural networks. By analyzing a biologically plausible model (ORGaNICs), we find that normalization allows circuits to remain stable even when recurrent interactions are strong enough to otherwise cause exploding neural dynamics. Furthermore, we identify a theoretical link between the breakdown of normalization and critical slowing down, a state where the brain recovers slowly from perturbations. This suggests that loss of normalization may serve as an early warning signal for the onset of pathological instabilities, such as seizures.
Related
- Shaping Slow Manifolds in Equivariant Recurrent Neural Networks
- Reusable modular architecture enables flexible cognitive operations in the mouse brain and artificial recurrent networks
- The ENL-USP7 complex regulates HIV latency through BRD4 protein stabilization
- Sampled-data exponential stabilization for delayed complex valued neural networks with quantized intermittent communication
- Adaptive Evolution of C4 Photosynthesis through Recurrent Lateral Gene Transfer
BibTeX
@article{morone2026stabilization,
title = {{Stabilization of recurrent neural networks through divisive normalization}},
author = {Flaviano Morone and Shivang Rawat and David J. Heeger and Stefano Martiniani},
journal = {Proceedings of the National Academy of Sciences},
year = {2026},
month = jul,
doi = {10.1073/pnas.2601841123},
url = {https://doi.org/10.1073/pnas.2601841123},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Stabilization of recurrent neural networks through divisive normalization
AU - Flaviano Morone
AU - Shivang Rawat
AU - David J. Heeger
AU - Stefano Martiniani
PY - 2026
DA - 2026/07/23
JO - Proceedings of the National Academy of Sciences
DO - 10.1073/pnas.2601841123
UR - https://doi.org/10.1073/pnas.2601841123
AB - Stability is a fundamental requirement for both biological and engineered neural circuits, yet it is surprisingly difficult to guarantee in the presence of recurrent interactions. Standard linear dynamical models of recurrent networks are sensitive to the precise values of the synaptic weights, since stability requires all eigenvalues of the recurrent matrix to lie within the unit circle. Here we demonstrate, both theoretically and numerically, that an arbitrary recurrent neural network can remain stable even when its spectral radius exceeds 1, provided it incorporates divisive normalization, a dynamical neural operation that suppresses the responses of individual neurons. Sufficiently strong recurrent weights lead to instability, but the approach to the unstable phase is preceded by a regime of critical slowing down, a well-known early warning signal for loss of stability. Remarkably, the onset of critical slowing down coincides with the breakdown of normalization, which we predict analytically as a function of the synaptic strength and the magnitude of the external input. Our findings suggest that the widespread implementation of normalization across neural systems may derive not only from its computational role, but also to enhance dynamical stability. I. SIGNIFICANCE STATEMENT Neural circuits must remain stable to function correctly, yet strong recurrent connections, which are essential for complex computations, often hinder this stability. We demonstrate that divisive normalization, a canonical neural operation found across many sensory systems, can actively stabilize neural networks. By analyzing a biologically plausible model (ORGaNICs), we find that normalization allows circuits to remain stable even when recurrent interactions are strong enough to otherwise cause exploding neural dynamics. Furthermore, we identify a theoretical link between the breakdown of normalization and critical slowing down, a state where the brain recovers slowly from perturbations. This suggests that loss of normalization may serve as an early warning signal for the onset of pathological instabilities, such as seizures.
SN - 0027-8424
SN - 1091-6490
ER -