Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
Abstract
A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non-linear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalization error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems.
Related
- Stimulus symmetries can confound representational similarity analyses
- COMPASS 2.0: psychometric representational similarity analysis distinguishes symptom structure from personal signal
- Multidimensional semantic representations emerge from multi-frequency representational similarity learning
- Decomposing deep neural network-brain representational similarity reveals distinct sources across the human visual hierarchy
- Assessing psychological, neural and AI-based representational geometries across multiple similarity properties
BibTeX
@misc{braun2026solutions,
title = {{Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks}},
author = {Lukas Braun and Erin Grant and Andrew M. Saxe},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38998},
eprint = {2609.38998},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38998},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Not all solutions are created equal: An analytical dissociation of functional and representational similarity in deep linear neural networks
AU - Lukas Braun
AU - Erin Grant
AU - Andrew M. Saxe
PY - 2026
DA - 2026/09/30
PB - arXiv
DO - 10.48550/arxiv.2609.38998
UR - https://arxiv.org/abs/2609.38998
AB - A foundational principle of connectionism is that perception, action, and cognition emerge from parallel computations among simple, interconnected units that generate and rely on neural representations. Accordingly, researchers employ multivariate pattern analysis to decode and compare the neural codes of artificial and biological networks, aiming to uncover their functions. However, there is limited analytical understanding of how a network's representation and function relate, despite this being essential to any quantitative notion of underlying function or functional similarity. We address this question using analysable two-layer linear networks and numerical simulations in non-linear networks. We find that function and representation are dissociated, allowing representational similarity without functional similarity and vice versa. Further, we show that neither robustness to input noise nor the level of generalization error constrain representations to the task. In contrast, networks robust to parameter noise have limited representational flexibility and must employ task-specific representations. Our findings suggest that representational alignment reflects computational advantages beyond functional alignment alone, with significant implications for interpreting and comparing the representations of connectionist systems.
ER -