Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience
Abstract
RSA and CKA are standard metrics for comparing neural representations across brain regions, organisms, and deep learning models. We demonstrate a fundamental weakness: these decoding-based metrics are insensitive to encoding manifold topology---the internal functional organization of a neural population. In a controlled MNIST experiment, RSA, CKA, and Procrustes R2 remain statistically unchanged when encoding topology is causally manipulated via an auxiliary clustering loss, while the two model populations differ significantly in attribution patterns, weight-graph assortativity, and out-of-distribution robustness. Across biological systems and machine learning models, similar decoding behavior can arise from small, non-representative subpopulations, and alignment metrics are insensitive to encoding manifold topology even when it is fundamentally altered. These findings bear directly on mechanistic interpretability: standard alignment metrics cannot distinguish whether two networks share the same computational circuits or merely produce indistinguishable aggregate outputs. We propose encoding manifolds and Gromov-Wasserstein distance as complementary diagnostics for any decoding-based similarity claim, and provide a Neural Manifold Explorer tool at https://johannesbertram.github.io/FNN_Manifolds/index.html for further experimentation.
Related
- Alignment of visual brain encoding models with individual perceptual similarity judgments
- EMAP-SSN: An Embedding- and Multiple-Alignment-Integrated Sequence Similarity Network Platform for Interactive Exploration of Protein Sequence Space
- Alignment-Aware Genomic Coordinate Encoding for Long-Range DNA Modeling
- Encoding and Decoding Temporal Signals with Spiking Bandpass Wavelets
- Scaling subjects in cross-modal alignment: video decoding with EEG foundation model
BibTeX
@misc{bertram2026decoding,
title = {{Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience}},
author = {Johannes Bertram and Luciano Dyballa and T. Anderson Keller and Savik Kinger and Steven W. Zucker},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.05907},
eprint = {2605.05907},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.05907},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Decoding Alignment without Encoding Alignment: A critique of similarity analysis in neuroscience
AU - Johannes Bertram
AU - Luciano Dyballa
AU - T. Anderson Keller
AU - Savik Kinger
AU - Steven W. Zucker
PY - 2026
DA - 2026/10/09
PB - arXiv
DO - 10.48550/arxiv.2605.05907
UR - https://arxiv.org/abs/2605.05907
AB - RSA and CKA are standard metrics for comparing neural representations across brain regions, organisms, and deep learning models. We demonstrate a fundamental weakness: these decoding-based metrics are insensitive to encoding manifold topology---the internal functional organization of a neural population. In a controlled MNIST experiment, RSA, CKA, and Procrustes R^2 remain statistically unchanged when encoding topology is causally manipulated via an auxiliary clustering loss, while the two model populations differ significantly in attribution patterns, weight-graph assortativity, and out-of-distribution robustness. Across biological systems and machine learning models, similar decoding behavior can arise from small, non-representative subpopulations, and alignment metrics are insensitive to encoding manifold topology even when it is fundamentally altered. These findings bear directly on mechanistic interpretability: standard alignment metrics cannot distinguish whether two networks share the same computational circuits or merely produce indistinguishable aggregate outputs. We propose encoding manifolds and Gromov-Wasserstein distance as complementary diagnostics for any decoding-based similarity claim, and provide a Neural Manifold Explorer tool at https://johannesbertram.github.io/FNN_Manifolds/index.html for further experimentation.
ER -