Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts
Abstract
Magnetic resonance imaging (MRI) of the brain routinely combines complementary contrasts—including T1-weighted (T1w), T2-weighted (T2w) and FLAIR sequences—to capture anatomy and pathology. Although foundation models trained on large neuroimaging datasets have shown promise, it remains unclear whether they effectively integrate these complementary contrasts across diverse clinical settings. Here we present Neuro-JEPA, a multimodal foundation model that learns unified representations across structural brain MRIs. Neuro-JEPA combines latent predictive learning with a sparse Mixture-of-Experts architecture to encode multiple MRI contrasts within a single model. The model is pretrained on 1.55 million scans from 428,647 clinical imaging studies spanning T1w, T2w and FLAIR sequences. The model performance is evaluated on 47 downstream tasks across three independent health systems and 12 public datasets, encompassing single-contrast, multimodal and cross-domain applications. Across these evaluations, Neuro-JEPA achieved the strongest overall performance and generalized robustly across heterogeneous cohorts, outperforming existing neuroimaging foundation models and strong conventional baselines in the majority of comparisons despite using less than one-third of the pretraining data of leading models. Performance gains were particularly pronounced in data-scarce settings, reaching up to 15% over prior foundation models for clinically important pathologies including gliosis and multiple sclerosis. Neuro-JEPA showed clinically relevant capabilities across domains, including prediction of amyloid status and progression from mild cognitive impairment to Alzheimer’s disease dementia, stroke identification and classification, and prediction of post-stroke functional outcome and length of stay. Together, these results show that data scale, curation and principled model design can act synergistically to produce broadly transferable neuroimaging representations, and establish Neuro-JEPA as a scalable framework toward general-purpose multimodal foundation models for neurological imaging.
Related
- An EfficientNet hybrid deep learning detection framework of network intrusion with an optimized mixture-of-experts architecture
- Asymmetric class-conditional routing in a federated quantum mixture-of-experts improves imbalanced intrusion detection in edge industrial IoT networks
- Evaluating Clinical Foundation Models for Early Alzheimer's Disease and Related Dementia Prediction from Longitudinal EHRs
- Building public health capacity: evaluating the intermediate Field Epidemiology Training Program of Papua New Guinea
- Evaluating the impact of the Hepatitis B Foundation’s international storytelling program on empowerment, advocacy, and stigma reduction
BibTeX
@misc{huang2026neurojepa,
title = {{Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts}},
author = {Haoxu Huang and Long Chen and Jingyun Chen and Jinu Hyun and James Loftus and Kara Melmed and Daniel Orringer and Jennifer Frontera and Seena Dehkharghani and Arjun Masurkar and Narges Razavian},
howpublished = {Research Square},
year = {2026},
month = oct,
doi = {10.21203/rs.3.rs-11130462/v1},
url = {https://doi.org/10.21203/rs.3.rs-11130462/v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Neuro-JEPA: Building and Evaluating Health-System Scale Neuroimaging Foundation Models with Mixture of Experts
AU - Haoxu Huang
AU - Long Chen
AU - Jingyun Chen
AU - Jinu Hyun
AU - James Loftus
AU - Kara Melmed
AU - Daniel Orringer
AU - Jennifer Frontera
AU - Seena Dehkharghani
AU - Arjun Masurkar
AU - Narges Razavian
PY - 2026
DA - 2026/10/15
PB - Research Square
DO - 10.21203/rs.3.rs-11130462/v1
UR - https://doi.org/10.21203/rs.3.rs-11130462/v1
AB - Magnetic resonance imaging (MRI) of the brain routinely combines complementary contrasts—including T1-weighted (T1w), T2-weighted (T2w) and FLAIR sequences—to capture anatomy and pathology. Although foundation models trained on large neuroimaging datasets have shown promise, it remains unclear whether they effectively integrate these complementary contrasts across diverse clinical settings. Here we present Neuro-JEPA, a multimodal foundation model that learns unified representations across structural brain MRIs. Neuro-JEPA combines latent predictive learning with a sparse Mixture-of-Experts architecture to encode multiple MRI contrasts within a single model. The model is pretrained on 1.55 million scans from 428,647 clinical imaging studies spanning T1w, T2w and FLAIR sequences. The model performance is evaluated on 47 downstream tasks across three independent health systems and 12 public datasets, encompassing single-contrast, multimodal and cross-domain applications. Across these evaluations, Neuro-JEPA achieved the strongest overall performance and generalized robustly across heterogeneous cohorts, outperforming existing neuroimaging foundation models and strong conventional baselines in the majority of comparisons despite using less than one-third of the pretraining data of leading models. Performance gains were particularly pronounced in data-scarce settings, reaching up to 15% over prior foundation models for clinically important pathologies including gliosis and multiple sclerosis. Neuro-JEPA showed clinically relevant capabilities across domains, including prediction of amyloid status and progression from mild cognitive impairment to Alzheimer’s disease dementia, stroke identification and classification, and prediction of post-stroke functional outcome and length of stay. Together, these results show that data scale, curation and principled model design can act synergistically to produce broadly transferable neuroimaging representations, and establish Neuro-JEPA as a scalable framework toward general-purpose multimodal foundation models for neurological imaging.
SN - 2693-5015
ER -