Many Brains, One Geometry: A Shared Visual-Semantic Space for Cross-Dataset fMRI Decoding
Abstract
Visual decoding from fMRI is typically siloed by participant and experiment, obscuring whether heterogeneous neural measurements can be organized within a common computational geometry. Here we introduce BRAID-fMRI (Brain Representation Alignment across Individuals and Datasets), a shared CLIP-supervised decoding framework. BRAID-fMRI uses a single ROI-wise Transformer with optional participant conditioning across eight visual-fMRI datasets comprising 93 dataset-specific participant entries, 430,007 single-trial responses and 162,839 unique stimuli. Regional brain activity is aligned with 512-dimensional CLIP ViT-B/32 representations using a multi-positive contrastive objective that treats repeated stimuli across participants and datasets as positives. BRAID-fMRI supports retrieval across seven evaluation datasets. On eight matched participant entries, it achieves 35.0 +/- 11.1% Top-10 accuracy, exceeding the observed mean accuracy of the two evaluated baselines - the MindEye-style pooled-CLIP decoder (27.1 +/- 5.1%) and ridge regression (21.1 +/- 9.1%) - and attaining the highest observed accuracy for seven of eight entries. In separately trained participant-agnostic models, expanding the source pool increased target-dataset holdout accuracy by up to 92.1% relative to the initial source-training condition. The learned space preserves graded semantic structure, while ablations and saliency highlight ventral and early visual cortex and category-specific motion and attentional systems. These results support scalable cross-dataset decoding into a common CLIP-aligned space, with model sensitivity concentrated in ventral and early visual inputs.
Related
- Multilingual Brain Decoding Reveals Shared Cortical Semantic Geometry
- Shared neural geometries for bilingual semantic representations in human hippocampal neurons
- Decoding and Characterizing the Intracranial Representation of Semantic Information
- Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech
- Shared latent representations of speech production for cross-patient speech decoding
BibTeX
@misc{khajehnejad2026many,
title = {{Many Brains, One Geometry: A Shared Visual-Semantic Space for Cross-Dataset fMRI Decoding}},
author = {Moein Khajehnejad and Michelangelo Tronti and Forough Habibollahi and Tommaso Boccato and Matteo Ferrante and Nicola Toschi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.09352},
eprint = {2610.09352},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.09352},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Many Brains, One Geometry: A Shared Visual-Semantic Space for Cross-Dataset fMRI Decoding
AU - Moein Khajehnejad
AU - Michelangelo Tronti
AU - Forough Habibollahi
AU - Tommaso Boccato
AU - Matteo Ferrante
AU - Nicola Toschi
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.09352
UR - https://arxiv.org/abs/2610.09352
AB - Visual decoding from fMRI is typically siloed by participant and experiment, obscuring whether heterogeneous neural measurements can be organized within a common computational geometry. Here we introduce BRAID-fMRI (Brain Representation Alignment across Individuals and Datasets), a shared CLIP-supervised decoding framework. BRAID-fMRI uses a single ROI-wise Transformer with optional participant conditioning across eight visual-fMRI datasets comprising 93 dataset-specific participant entries, 430,007 single-trial responses and 162,839 unique stimuli. Regional brain activity is aligned with 512-dimensional CLIP ViT-B/32 representations using a multi-positive contrastive objective that treats repeated stimuli across participants and datasets as positives. BRAID-fMRI supports retrieval across seven evaluation datasets. On eight matched participant entries, it achieves 35.0 +/- 11.1% Top-10 accuracy, exceeding the observed mean accuracy of the two evaluated baselines - the MindEye-style pooled-CLIP decoder (27.1 +/- 5.1%) and ridge regression (21.1 +/- 9.1%) - and attaining the highest observed accuracy for seven of eight entries. In separately trained participant-agnostic models, expanding the source pool increased target-dataset holdout accuracy by up to 92.1% relative to the initial source-training condition. The learned space preserves graded semantic structure, while ablations and saliency highlight ventral and early visual cortex and category-specific motion and attentional systems. These results support scalable cross-dataset decoding into a common CLIP-aligned space, with model sensitivity concentrated in ventral and early visual inputs.
ER -