FERMAT: An Autoregressive Clinical Foundation Model for Disease Risk and Clinical Event Prediction
- 1Seoul National University
Abstract
Clinical foundation models learn patterns in longitudinal health records that can be reused across prediction tasks. However, the relative strengths of representation-based and generation-based prediction across clinical tasks remain unclear. Using longitudinal electronic health records from 3.77 million patients at a single hospital, we developed FERMAT (Foundation model for Exploring Real-world Multimodal health data using Autoregressive Trajectory modeling), a 65-million-parameter clinical foundation model. We compared predictions from task-specific predictors trained on patient representations with those derived from generated clinical event sequences. Representation-based prediction performed better in the disease-risk comparisons. In detailed analyses of five-year chronic kidney disease, fatty liver, and diabetes risk in a test cohort of 10,000 patients, representation-based predictors exceeded generation-based estimates by 0.098-0.115 in observation-weighted AUROC and had lower probability error. Generation-based prediction, in turn, performed better in predicting diagnoses, prescriptions, and procedures recorded on the same day, a task motivated by the frequent presence of multiple events with the same date in hospital EHRs. Using 50 generated trajectories per patient, it achieved a Recall@10 of 71.0%, compared with 55.7% for representation-based prediction, with a paired improvement of 15.31 percentage points (95% CI, 13.92-16.79; n = 1,862). These findings demonstrate task-dependent strengths of representation-based and generation-based applications within a single clinical foundation model, providing an empirical basis for exploring how the two approaches can be used individually or combined to address clinical prediction needs.
Related
- Evaluating Clinical Foundation Models for Early Alzheimer's Disease and Related Dementia Prediction from Longitudinal EHRs
- Nomogram models integrating multiregion radiomics, quantitative computed tomography, and clinical indicators for distinguishing connective tissue disease-associated interstitial lung disease from idiopathic pulmonary fibrosis and improving the prediction of gender-age-physiology stage
- No evidence that serum metabolite slopes improve held-out prediction of clinical progression in the Alzheimer’s disease neuroimaging initiative
- Prediction of cognitive impairment in Parkinson’s disease using TabPFN: an integrative analysis of clinical data, longitudinal trajectories, and lipidomics
- Development of a parallel cascaded adaptive ensemble framework with feature engineering process for the prediction of neurodegenerative disease using temporal clinical data
BibTeX
@misc{jung2026fermat,
title = {{FERMAT: An Autoregressive Clinical Foundation Model for Disease Risk and Clinical Event Prediction}},
author = {J. Jung and H.-C. Lee},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.11.26365210},
url = {https://www.medrxiv.org/content/10.64898/2026.10.11.26365210v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - FERMAT: An Autoregressive Clinical Foundation Model for Disease Risk and Clinical Event Prediction
AU - J. Jung
AU - H.-C. Lee
PY - 2026
DA - 2026/10/11
PB - medRxiv
DO - 10.64898/2026.10.11.26365210
UR - https://www.medrxiv.org/content/10.64898/2026.10.11.26365210v1
AB - Clinical foundation models learn patterns in longitudinal health records that can be reused across prediction tasks. However, the relative strengths of representation-based and generation-based prediction across clinical tasks remain unclear. Using longitudinal electronic health records from 3.77 million patients at a single hospital, we developed FERMAT (Foundation model for Exploring Real-world Multimodal health data using Autoregressive Trajectory modeling), a 65-million-parameter clinical foundation model. We compared predictions from task-specific predictors trained on patient representations with those derived from generated clinical event sequences. Representation-based prediction performed better in the disease-risk comparisons. In detailed analyses of five-year chronic kidney disease, fatty liver, and diabetes risk in a test cohort of 10,000 patients, representation-based predictors exceeded generation-based estimates by 0.098-0.115 in observation-weighted AUROC and had lower probability error. Generation-based prediction, in turn, performed better in predicting diagnoses, prescriptions, and procedures recorded on the same day, a task motivated by the frequent presence of multiple events with the same date in hospital EHRs. Using 50 generated trajectories per patient, it achieved a Recall@10 of 71.0%, compared with 55.7% for representation-based prediction, with a paired improvement of 15.31 percentage points (95% CI, 13.92-16.79; n = 1,862). These findings demonstrate task-dependent strengths of representation-based and generation-based applications within a single clinical foundation model, providing an empirical basis for exploring how the two approaches can be used individually or combined to address clinical prediction needs.
ER -