Neural networks as decision trees: an analytical solution for learning and neural selectivity
Abstract
Nonlinear neural networks develop structured internal representations, yet how their geometry is determined by the tasks being learned remains poorly understood. Here, we develop an analytical framework for piecewise-linear feedforward and recurrent networks that links learning, activation-region structure, and neural selectivity. We show that, at any stationary point of gradient-aligned learning, a nonlinear network decomposes into local linear regressions over its activation regions. Deviations from the corresponding least-squares solutions are jointly constrained by network weights shared across regions and vanish in the low-error regime, yielding an approximate piecewise least-squares decomposition of the task. This structure admits a decision-tree interpretation, which we recover numerically by fitting trees to predict network activation patterns from the input. We further derive how the regions in which neurons are active determine the task statistics they capture, thereby organizing neural selectivity into distinct subpopulations. The resulting predictions closely match the selectivity geometry observed in simulated networks and two empirical neural datasets. Finally, we show that neural baseline regulates activation pattern diversity, placing networks along a continuum between coarse, generalizing representations and fine-grained, expressive representations. Together, these results establish activation regions as a unifying framework for describing how nonlinear networks decompose task structure into local computations and how those computations can be revealed through neural activity.
Related
- Neural networks as decision trees: an analytical framework for learning and neural selectivity
- Selectivity Dilution in Three-Factor Learning: A Finite-Population Model with Potential Implications for Motor Development and Post-Stroke Motor Rehabilitation
- Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
- Exploring the role of multivariate associations between brain networks and gut microbiota in (food) value-based decision-making
- Efficiency and robustness partition the solution space for memory in recurrent networks
BibTeX
@misc{tissot2026neural,
title = {{Neural networks as decision trees: an analytical solution for learning and neural selectivity}},
author = {Hugo Tissot and Jonas Ranft and Yves Boubenec},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.08228},
eprint = {2610.08228},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.08228},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Neural networks as decision trees: an analytical solution for learning and neural selectivity
AU - Hugo Tissot
AU - Jonas Ranft
AU - Yves Boubenec
PY - 2026
DA - 2026/10/06
PB - arXiv
DO - 10.48550/arxiv.2610.08228
UR - https://arxiv.org/abs/2610.08228
AB - Nonlinear neural networks develop structured internal representations, yet how their geometry is determined by the tasks being learned remains poorly understood. Here, we develop an analytical framework for piecewise-linear feedforward and recurrent networks that links learning, activation-region structure, and neural selectivity. We show that, at any stationary point of gradient-aligned learning, a nonlinear network decomposes into local linear regressions over its activation regions. Deviations from the corresponding least-squares solutions are jointly constrained by network weights shared across regions and vanish in the low-error regime, yielding an approximate piecewise least-squares decomposition of the task. This structure admits a decision-tree interpretation, which we recover numerically by fitting trees to predict network activation patterns from the input. We further derive how the regions in which neurons are active determine the task statistics they capture, thereby organizing neural selectivity into distinct subpopulations. The resulting predictions closely match the selectivity geometry observed in simulated networks and two empirical neural datasets. Finally, we show that neural baseline regulates activation pattern diversity, placing networks along a continuum between coarse, generalizing representations and fine-grained, expressive representations. Together, these results establish activation regions as a unifying framework for describing how nonlinear networks decompose task structure into local computations and how those computations can be revealed through neural activity.
ER -