Efficiency and robustness partition the solution space for memory in recurrent networks
Abstract
In computational neuroscience, task-trained recurrent neural networks (RNNs) are commonly used as a testbed for exploring the space of recurrent circuit solutions compatible with a particular function. However, these networks are subject to inductive biases that may be misaligned with those of biological circuits, which operate under various efficiency and robustness constraints. Here, using a minimal stimulus recall task, we systematically characterize how the solution space of recurrent neural networks is shaped by the desiderata of weight efficiency, activity efficiency, and robustness to noise. We show that weight efficient solutions exhibit a low-rank bias that aligns with the inductive biases of task-trained networks and favors dynamical motifs involving slow, persistent dynamics, whereas activity efficiency encourages high-rank solutions that rely on transient amplification. Moreover, we find that activity efficient solutions exhibit a strong dissociation between high-variance modes of activity and those that causally influence network output. Unlike in static feedforward settings, we demonstrate that noise robustness and activity efficiency are generally in opposition, rather than synergistic. Using these findings, we propose a normative account of misaligned coding phenomena recently identified in premotor cortex.
Related
- A Reexamination of Noise-Driven Robustness Evolution in Gene Regulatory Networks
- Reusable modular architecture enables flexible cognitive operations in the mouse brain and artificial recurrent networks
- Assessing the robustness of SNaQ to violations induced by high-level phylogenetic networks
- Fundamental Bounds on the Oscillation Efficiency of Markov Networks
- Heterogeneous plasticity rules generate diverse structures in recurrent networks
BibTeX
@misc{qian2026efficiency,
title = {{Efficiency and robustness partition the solution space for memory in recurrent networks}},
author = {William Qian and Cengiz Pehlevan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04697},
eprint = {2610.04697},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04697},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Efficiency and robustness partition the solution space for memory in recurrent networks
AU - William Qian
AU - Cengiz Pehlevan
PY - 2026
DA - 2026/10/03
PB - arXiv
DO - 10.48550/arxiv.2610.04697
UR - https://arxiv.org/abs/2610.04697
AB - In computational neuroscience, task-trained recurrent neural networks (RNNs) are commonly used as a testbed for exploring the space of recurrent circuit solutions compatible with a particular function. However, these networks are subject to inductive biases that may be misaligned with those of biological circuits, which operate under various efficiency and robustness constraints. Here, using a minimal stimulus recall task, we systematically characterize how the solution space of recurrent neural networks is shaped by the desiderata of weight efficiency, activity efficiency, and robustness to noise. We show that weight efficient solutions exhibit a low-rank bias that aligns with the inductive biases of task-trained networks and favors dynamical motifs involving slow, persistent dynamics, whereas activity efficiency encourages high-rank solutions that rely on transient amplification. Moreover, we find that activity efficient solutions exhibit a strong dissociation between high-variance modes of activity and those that causally influence network output. Unlike in static feedforward settings, we demonstrate that noise robustness and activity efficiency are generally in opposition, rather than synergistic. Using these findings, we propose a normative account of misaligned coding phenomena recently identified in premotor cortex.
ER -