BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models
Abstract
fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.
Related
- What masking geometry works best for EEG foundation models?
- Single-cell foundation models transfer across deep evolutionary distances
- Prediction transfer does not imply reliability transfer in single-cell foundation models
- Lightweight Semantic EEG Foundation Model for Frozen Cross-Disorder Transfer
- From Simulated Fluids to Living Tissue: Zero-Shot Foundation Model Transfer to Perturbed Embryos
BibTeX
@misc{xia2026braintaskonomy,
title = {{BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models}},
author = {Junfeng Xia and Wenhao Ye and Junxiang Zhang and Jiayu Zuo and Mo Wang and Quanying Liu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.10518},
eprint = {2609.10518},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.10518},
}
RIS
TY - UNPB
TI - BrainTaskonomy: Learning How to Pretrain and What to Transfer in fMRI Foundation Models
AU - Junfeng Xia
AU - Wenhao Ye
AU - Junxiang Zhang
AU - Jiayu Zuo
AU - Mo Wang
AU - Quanying Liu
PY - 2026
DA - 2026/09/09
PB - arXiv
DO - 10.48550/arxiv.2609.10518
UR - https://arxiv.org/abs/2609.10518
AB - fMRI foundation models increasingly aggregate heterogeneous data across brain states, cohorts, and acquisition settings, yet pretraining domains are commonly treated as a flat mixture and downstream tasks are adapted independently. We study whether measured learning relations can organize both stages without modifying the backbone. During pretraining, a lightweight Brain-DiT proxy estimates difficulty and directed facilitation across ten fMRI domains, yielding a priority-guided cumulative domain curriculum combined with high-to-low-noise timestep scheduling and joint consolidation. During adaptation, controlled first- and higher-order transfer across fifteen tasks constructs a directed taskonomy, from which budgeted integer programming (BIP) selects directly supervised source tasks and target-specific routes. The joint priority-domain and high-to-low-timestep curriculum reduces v-NMSE, PSD-NMSE, and FC-MSE by 6.5%, 16.3%, and 10.5%, respectively, relative to uniform sampling over both dimensions, and shows strong downstream performance across six in- and out-of-domain tasks. The taskonomy reveals asymmetric, target-dependent transfer, while exploratory sealed-test evaluation shows larger descriptive gains for BIP policies when higher-order route spaces are available than for matched random controls. Together, these findings support organizing fMRI pretraining and adaptation by measured learning relations rather than treating domains and tasks as independent flat sets.
ER -