Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling
Abstract
Pruning network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. We evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses connection magnitude and exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a threshold, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average strength. We show that this sampling and rescaling is essential to good performance, and can augment other pruning strategies. We also find that the optimal empirical degree of rescaling is lower than predicted by the original theoretical argument, and thus develop a new version of noise-prune that outperforms the previous version. Our results demonstrate that a biologically-motivated pruning strategy based on local activity statistics can effectively preserve task performance in functional recurrent networks by combining activity-dependent connection selection with compensatory strengthening of surviving connections.
Related
- "More Is Different'' in Neural Circuits: Algebraic Emergence of Effective Theories in Canonical Recurrent Motifs of Biological Neuronal Networks
- Seasonal fluctuations in fitness result in severe reductions in effective population size
- Shaping Slow Manifolds in Equivariant Recurrent Neural Networks
- Reusable modular architecture enables flexible cognitive operations in the mouse brain and artificial recurrent networks
- Heterogeneous plasticity rules generate diverse structures in recurrent networks
BibTeX
@misc{senthil2026effective,
title = {{Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling}},
author = {Sanjith Senthil and Rishidev Chaudhuri},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2608.05464},
eprint = {2608.05464},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.05464},
}
RIS
TY - UNPB
TI - Effective pruning of task-trained recurrent neural networks using noisy fluctuations and connection rescaling
AU - Sanjith Senthil
AU - Rishidev Chaudhuri
PY - 2026
DA - 2026/11/04
PB - arXiv
DO - 10.48550/arxiv.2608.05464
UR - https://arxiv.org/abs/2608.05464
AB - Pruning network connections is key to brain function but, despite its importance, there exist few biologically-plausible pruning rules with demonstrated good performance. We evaluate noise-prune, a recently introduced unsupervised local pruning rule for recurrent networks that uses noisy fluctuations to determine the importance of connections. Noise-prune has previously only been empirically tested on random networks without a specific computational function. We show that noise-prune preserves task-performance in task-trained recurrent neural networks, greatly outperforming a strategy that only uses connection magnitude and exceeding a non-local strategy that uses second-order information. Rather than deterministically removing connections that fall below a threshold, noise-prune samples connections to preserve based on their importance and strengthens retained connections to preserve average strength. We show that this sampling and rescaling is essential to good performance, and can augment other pruning strategies. We also find that the optimal empirical degree of rescaling is lower than predicted by the original theoretical argument, and thus develop a new version of noise-prune that outperforms the previous version. Our results demonstrate that a biologically-motivated pruning strategy based on local activity statistics can effectively preserve task performance in functional recurrent networks by combining activity-dependent connection selection with compensatory strengthening of surviving connections.
ER -