Beyond Marginal Coverage: Class-Conditional Conformal Prediction in Multi-Class Psychiatric Neuroimaging
- 1Gerogia State University
Abstract
Conformal prediction is increasingly proposed for clinical decision support because it provides distribution-free coverage guarantees that hold for any model and any data distribution. The guarantee routinely reported, however, is marginal, and we show that in multi-class diagnostic classification it can be satisfied exactly while individual diagnoses are covered very unevenly. On a four-way mood and psychosis classification task over 1,520 subjects from three studies and 14 acquisition sites, marginal split-conformal calibration attained 0.9000 empirical coverage against a nominal 0.90, while healthy controls were covered at 0.941 and schizoaffective disorder at 0.818. The surplus and the deficit cancel, so the aggregate figure is uninformative about either. Class-conditional (Mondrian) calibration reduced this 12.3-point disparity to 0.4 points at a cost of 0.07 labels in mean set size, under 3%. Holding coverage fixed across diagnoses has a further consequence: the residual variation in set size can no longer be attributed to class prior or per-class accuracy, and becomes interpretable as a property of the subject. We show that the resulting prediction sets separate subjects into confident, boundary, ambiguous and unresolved strata, and that the proportion of subjects independently flagged as label-ambiguous by a structural-MRI model trained separately on the same cohort rises monotonically across these strata (34.1%, 57.3%, 68.1%, 81.8%; p = 8.8e-18). No schizoaffective subject and 0.9% of bipolar subjects reach the confident stratum, against 18.0% of controls and 13.4% of schizophrenia subjects. Set size at matched coverage also provides a comparison between representations that accuracy cannot. Applied to structural MRI, functional MRI and their fusion, it reveals a sign reversal that accuracy conceals: fusion reduces set size for bipolar (-0.25) and schizoaffective (-0.23) subjects and increases it for controls (+0.16), and the aggregate fusion benefit itself changes sign below alpha = 0.10, while top-1 accuracy rises monotonically from 0.495 to 0.578 to 0.611 across the three models. Reporting marginal coverage alone is insufficient whenever diagnostic classes are unbalanced.
Related
- Supervised Domain Adaptation Mitigates Cross-Ethnicity Prediction Error in Neuroimaging Based Cognitive Prediction
- PerturbBridge: Conditional Latent Schr\"{o}dinger Bridge for Single-Cell Perturbation Prediction
- Representation-aware conformal prediction for reliable uncertainty quantification in drug-target affinity
- From {triangleup}{triangleup}G Prediction to Mutation Prioritization with Conformal Prediction
- OmniTCR: a foundation model unifying T cell receptor recognition prediction and conditional sequence generation
BibTeX
@misc{rokham2026beyond,
title = {{Beyond Marginal Coverage: Class-Conditional Conformal Prediction in Multi-Class Psychiatric Neuroimaging}},
author = {H. Rokham and H. Falakshahi and V. Calhoun},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.23.753958},
url = {https://www.biorxiv.org/content/10.64898/2026.09.23.753958v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Beyond Marginal Coverage: Class-Conditional Conformal Prediction in Multi-Class Psychiatric Neuroimaging
AU - H. Rokham
AU - H. Falakshahi
AU - V. Calhoun
PY - 2026
DA - 2026/09/23
PB - bioRxiv
DO - 10.64898/2026.09.23.753958
UR - https://www.biorxiv.org/content/10.64898/2026.09.23.753958v1
AB - Conformal prediction is increasingly proposed for clinical decision support because it provides distribution-free coverage guarantees that hold for any model and any data distribution. The guarantee routinely reported, however, is marginal, and we show that in multi-class diagnostic classification it can be satisfied exactly while individual diagnoses are covered very unevenly. On a four-way mood and psychosis classification task over 1,520 subjects from three studies and 14 acquisition sites, marginal split-conformal calibration attained 0.9000 empirical coverage against a nominal 0.90, while healthy controls were covered at 0.941 and schizoaffective disorder at 0.818. The surplus and the deficit cancel, so the aggregate figure is uninformative about either. Class-conditional (Mondrian) calibration reduced this 12.3-point disparity to 0.4 points at a cost of 0.07 labels in mean set size, under 3%. Holding coverage fixed across diagnoses has a further consequence: the residual variation in set size can no longer be attributed to class prior or per-class accuracy, and becomes interpretable as a property of the subject. We show that the resulting prediction sets separate subjects into confident, boundary, ambiguous and unresolved strata, and that the proportion of subjects independently flagged as label-ambiguous by a structural-MRI model trained separately on the same cohort rises monotonically across these strata (34.1%, 57.3%, 68.1%, 81.8%; p = 8.8e-18). No schizoaffective subject and 0.9% of bipolar subjects reach the confident stratum, against 18.0% of controls and 13.4% of schizophrenia subjects. Set size at matched coverage also provides a comparison between representations that accuracy cannot. Applied to structural MRI, functional MRI and their fusion, it reveals a sign reversal that accuracy conceals: fusion reduces set size for bipolar (-0.25) and schizoaffective (-0.23) subjects and increases it for controls (+0.16), and the aggregate fusion benefit itself changes sign below alpha = 0.10, while top-1 accuracy rises monotonically from 0.495 to 0.578 to 0.611 across the three models. Reporting marginal coverage alone is insufficient whenever diagnostic classes are unbalanced.
ER -