Prospective validation of an uncertainty-aware machine learning model for secondary progressive multiple sclerosis classification
- 1Uppsala University
Abstract
BackgroundRelapsing-remitting multiple sclerosis (RRMS) often progresses to secondary-progressive MS (SPMS), but diagnosis is typically retrospective, delaying potential interventions. Existing rule-based and machine learning approaches often lack prospective validation and provide no uncertainty estimates, limiting clinical adoption.
ObjectiveTo prospectively evaluate a conformal prediction (CP)-enabled random forest model for reproducing clinician-recorded disease course labels in the Swedish Multiple Sclerosis Registry (SMSReg) and compare its classifications with the Lorscheider criteria.
MethodsThe CP model was trained and calibrated on SMSReg-2022 (10,787 individuals, 85,461 hospital visits) and then validated on SMSReg-2025 (15,719 individuals, 45,077 hospital visits; not included in the SMSReg-2022 training/calibration/testing datasets). Performance was assessed at the hospital visit and individual levels and compared with the Lorscheider criteria.
ResultsThe model maintained good calibration on the prospective dataset. At the per-hospital visit-level, the model accuracies were 92% (RRMS) and 95% (SPMS). At the individual level, sensitivity was 98%, specificity 89%, and F1-score 0.75. With clinician-recorded labels as reference, the model showed higher sensitivity than the Lorscheider criteria (98% vs. 71%), more transition assignments (90% vs. 57%), and higher F1-score (0.76 vs. 0.69).
ConclusionThe CP-enabled model prospectively reproduced clinician-recorded disease course labels with calibrated uncertainty, supporting harmonization and decision support.
Related
- Prospective External Validation of the HFA-ICOS Baseline Cardiovascular Risk Classification in Women with Breast Cancer Receiving Anthracycline-Containing Therapy in a Central Asian Cohort
- Chronic lesion expansion persists in secondary progressive multiple sclerosis and is associated with neurodegeneration
- Rapidly Progressive Buschke-Löwenstein Tumor Managed With Wide Local Excision and Healing by Secondary Intention: A Case Report and Literature Review.
- Assessment of functional profile and fatigue in Egyptian patients with active secondary progressive MS
- Plasma biomarker strategies to reduce diagnostic uncertainty in p-tau217–based assessment of CSF amyloid positivity in secondary-care patients with MCI
BibTeX
@misc{sreenivasan2026prospective,
title = {{Prospective validation of an uncertainty-aware machine learning model for secondary progressive multiple sclerosis classification}},
author = {A. Parakkal Sreenivasan and Y. Noui and I. Erngren and O. Spjuth and J. Burman and K. Kultima},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.07.26364939},
url = {https://www.medrxiv.org/content/10.64898/2026.10.07.26364939v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Prospective validation of an uncertainty-aware machine learning model for secondary progressive multiple sclerosis classification
AU - A. Parakkal Sreenivasan
AU - Y. Noui
AU - I. Erngren
AU - O. Spjuth
AU - J. Burman
AU - K. Kultima
PY - 2026
DA - 2026/10/07
PB - medRxiv
DO - 10.64898/2026.10.07.26364939
UR - https://www.medrxiv.org/content/10.64898/2026.10.07.26364939v1
AB - BackgroundRelapsing-remitting multiple sclerosis (RRMS) often progresses to secondary-progressive MS (SPMS), but diagnosis is typically retrospective, delaying potential interventions. Existing rule-based and machine learning approaches often lack prospective validation and provide no uncertainty estimates, limiting clinical adoption. ObjectiveTo prospectively evaluate a conformal prediction (CP)-enabled random forest model for reproducing clinician-recorded disease course labels in the Swedish Multiple Sclerosis Registry (SMSReg) and compare its classifications with the Lorscheider criteria. MethodsThe CP model was trained and calibrated on SMSReg-2022 (10,787 individuals, 85,461 hospital visits) and then validated on SMSReg-2025 (15,719 individuals, 45,077 hospital visits; not included in the SMSReg-2022 training/calibration/testing datasets). Performance was assessed at the hospital visit and individual levels and compared with the Lorscheider criteria. ResultsThe model maintained good calibration on the prospective dataset. At the per-hospital visit-level, the model accuracies were 92% (RRMS) and 95% (SPMS). At the individual level, sensitivity was 98%, specificity 89%, and F1-score 0.75. With clinician-recorded labels as reference, the model showed higher sensitivity than the Lorscheider criteria (98% vs. 71%), more transition assignments (90% vs. 57%), and higher F1-score (0.76 vs. 0.69). ConclusionThe CP-enabled model prospectively reproduced clinician-recorded disease course labels with calibrated uncertainty, supporting harmonization and decision support.
ER -