Large language models poorly capture how word order shapes representations of sentence meaning in the human brain.
- 1The Centre for Brain, Mind and Markets, Faculty of Business and Economics, The University of Melbourne, Melbourne, Australia.
- 2Faculty of Social Data Science, Hitotsubashi University, Kunitachi, Japan.
Abstract
Large language models based on the transformer architecture are now capable of producing human-like language. But do they encode and process linguistic meaning in a human-like way? Here, we address this question by analysing 7T fMRI data from 30 participants reading 108 sentences each. These sentences are carefully designed to disentangle sentence structure from word meaning, thereby testing whether transformer representations of sentence meaning resemble those formed by the brain. Using representational similarity analysis, we compared representations from eight transformers and eleven other semantic models to brain activity in language regions of interest. We found that while transformer models match brain representations better than models that completely ignore word order, all transformers poorly match human brain representations of sentences that require sensitivity to structural relations such as semantic roles (who did what to whom). Further, transformers were significantly inferior to models explicitly designed to encode the structural relations between words. Our results provide insight into the nature of sentence representation in the brain, highlighting the critical role of sentence structure. They also cast doubt on the claim that transformers represent sentence meaning similarly to the human brain.
Related
- Semantic ambiguity dissociates the roles of word meaning and co-occurrence statistics in sentence processing
- Prioritizing meaning: value, prediction, and age differences in sentence processing
- Not that cloze: Sentence constraint is influenced by language and language dominance.
- Quantifying context-dependent meaning representations: a novel procedure
- When meaning becomes decodable: Linking neural markers of semantic processing and semantic representations
BibTeX
@article{fodor2026large,
title = {{Large language models poorly capture how word order shapes representations of sentence meaning in the human brain.}},
author = {James Fodor and Carsten Murawski and Shinsuke Suzuki},
journal = {ELife},
year = {2026},
month = oct,
doi = {10.7554/elife.108442},
url = {https://europepmc.org/article/MED/42924585},
note = {CC BY},
}
RIS
TY - JOUR
TI - Large language models poorly capture how word order shapes representations of sentence meaning in the human brain.
AU - James Fodor
AU - Carsten Murawski
AU - Shinsuke Suzuki
PY - 2026
DA - 2026/10/23
JO - ELife
DO - 10.7554/elife.108442
UR - https://europepmc.org/article/MED/42924585
AB - Large language models based on the transformer architecture are now capable of producing human-like language. But do they encode and process linguistic meaning in a human-like way? Here, we address this question by analysing 7T fMRI data from 30 participants reading 108 sentences each. These sentences are carefully designed to disentangle sentence structure from word meaning, thereby testing whether transformer representations of sentence meaning resemble those formed by the brain. Using representational similarity analysis, we compared representations from eight transformers and eleven other semantic models to brain activity in language regions of interest. We found that while transformer models match brain representations better than models that completely ignore word order, all transformers poorly match human brain representations of sentences that require sensitivity to structural relations such as semantic roles (who did what to whom). Further, transformers were significantly inferior to models explicitly designed to encode the structural relations between words. Our results provide insight into the nature of sentence representation in the brain, highlighting the critical role of sentence structure. They also cast doubt on the claim that transformers represent sentence meaning similarly to the human brain.
SN - 2050-084X
ER -