The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer
Abstract
Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.
Related
- Approximate local alignment via chained MEM divergence estimation for detecting horizontal gene transfer
- Alignment with sex-typical brain architecture: Links to mood pathology, sex hormones and transcriptomics of neurodegenerative conditions
- Regional alignment with sex-typical brain architecture, maternal health and environmental enrichment predict psychiatric symptoms in early life
- Two opposite failure modes in alignment-based conservation scores, one inflating and one concealing constraint, are computable before the score is used
- What Shapes Brain Alignment of Large Language Models during Dialogue? Insights from Language Production and Comprehension
BibTeX
@misc{rahbar2026score,
title = {{The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer}},
author = {Saman Rahbar},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.03827},
eprint = {2610.03827},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.03827},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The Score Is Not the Structure: Brain Alignment and Cross-Lingual Transfer
AU - Saman Rahbar
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.03827
UR - https://arxiv.org/abs/2610.03827
AB - Similarity scores are often offered as evidence that a model shares structure with the brain or across languages. We ask what such a score reads when that structure is removed, or when the instrument measuring it does not work, in two settings. Across seventeen languages, a grammaticality probe transfers worse between more distant languages (r = -0.66). But the probe's own accuracy falls along the same axis and is at chance in four languages, four of the five most distant. Dropping them halves the explained variance, and because it also narrows the range of distances, the design cannot say how much of the gradient is the instrument. Counting the 272 language pairs as independent gives p = 0.0006 for a steering effect that is null when the seventeen languages are the unit (p = 0.155). In brain alignment, a training objective raises a language model's similarity to fMRI responses from 0.10 to 0.34 (reliability ceiling 0.54), yet targets with the brain correspondence destroyed still reach 0.31. A shuffled target scores about k/n against the real one, for rank k and n sentences, a baseline that can be computed before any model is trained. Both interventions work, yet we detect no distance-graded steering effect and no syntactic benefit from alignment. Before reading a correspondence score, measure how much of it survives without the correspondence.
ER -