Foundation Models as Modular Tools for fMRI Analysis
Abstract
Foundation models (FMs) for the brain are rapidly growing in number and each captures a different aspect of the same data. A single scientific question often needs several of them at once, alongside classical analysis and the literature, but no established method combines them, because each model defines its own inputs, outputs, and spatial conventions. Representation alignment, the usual route from an FM into a language model, binds one FM to one LLM and requires retraining for every model added. We take a modular route: each FM becomes an independent tool server that a fixed language agent calls next to classical and literature tools, with no retraining. We start with fMRI, where public FMs already disagree on how to name a brain location and open task data give a classical baseline, and serve two FMs at opposite ends of that spectrum, BrainLM (atlas parcels) and Omni-fMRI (atlas-free voxels). A 76-question benchmark, a 2 by 2 tool ablation, and an independent LLM judge yield 3,040 judged traces. In this first study, modular FM access raises holistic answer quality by +0.70 on a 1 to 5 rubric (95% CI [+0.44, +0.97], n=34). The gain lands in how the agent selects, parameterizes, and grounds tool calls, and peaks where the agent must cross FM vocabularies. The two FMs win on different questions, yet mounting both matches the better one, and the voxel interface fails 7.8 times more often by context overflow. Modular connection works. Orchestrating what it connects, across EEG and MRI models, is the work ahead.
Related
- FlatClip: Reusing Image Foundation Models for fMRI Representation Learning via Cortical Flatmaps
- BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models
- Transformers for fMRI analysis: from spatiotemporal modeling to brain foundation models.
- PRK introns provide modular tools for enhanced transgene expression in Chlamydomonas reinhardtii
- gcat, its_a_scam: modular tools for single cell data processing and per-cellquality control
BibTeX
@misc{kilgore2026foundation,
title = {{Foundation Models as Modular Tools for fMRI Analysis}},
author = {Joseph Kilgore and Noemia Mahmud and Shinjae Yoo and David Keetae Park},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.06926},
eprint = {2611.06926},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.06926},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Foundation Models as Modular Tools for fMRI Analysis
AU - Joseph Kilgore
AU - Noemia Mahmud
AU - Shinjae Yoo
AU - David Keetae Park
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.06926
UR - https://arxiv.org/abs/2611.06926
AB - Foundation models (FMs) for the brain are rapidly growing in number and each captures a different aspect of the same data. A single scientific question often needs several of them at once, alongside classical analysis and the literature, but no established method combines them, because each model defines its own inputs, outputs, and spatial conventions. Representation alignment, the usual route from an FM into a language model, binds one FM to one LLM and requires retraining for every model added. We take a modular route: each FM becomes an independent tool server that a fixed language agent calls next to classical and literature tools, with no retraining. We start with fMRI, where public FMs already disagree on how to name a brain location and open task data give a classical baseline, and serve two FMs at opposite ends of that spectrum, BrainLM (atlas parcels) and Omni-fMRI (atlas-free voxels). A 76-question benchmark, a 2 by 2 tool ablation, and an independent LLM judge yield 3,040 judged traces. In this first study, modular FM access raises holistic answer quality by +0.70 on a 1 to 5 rubric (95% CI [+0.44, +0.97], n=34). The gain lands in how the agent selects, parameterizes, and grounds tool calls, and peaks where the agent must cross FM vocabularies. The two FMs win on different questions, yet mounting both matches the better one, and the voxel interface fails 7.8 times more often by context overflow. Modular connection works. Orchestrating what it connects, across EEG and MRI models, is the work ahead.
ER -