Ensemble SHAP Aggregation and Attribution Variability in Clinical Machine Learning: A COVID-19 Mortality Study
- 1Global Health Institute, Michigan State University, MI, USA
Abstract
ObjectiveTo combine performance-weighted ensemble SHAP aggregation with resampling-based assessment of feature-importance variability and patient-level attribution alignment for interpreting COVID-19 mortality predictions.
MethodsWe analyzed a prospective cohort of 1,857 patients hospitalized with COVID-19 at two hospitals in Peru. Ten predictive algorithms were evaluated using five-fold cross-validation, and their mean AUROC values determined their attribution-aggregation weights. Within each model and training trial, signed SHAP values were normalized by the mean total absolute attribution across evaluation patients before performance-weighted aggregation. Feature importance was summarized across 30 resampled training trials using means and normal-approximation 95% confidence intervals. Patient-level attribution alignment was assessed using cosine similarity between signed feature-attribution vectors for corresponding patients, with whole-trial patient-correspondence randomization and Benjamini-Hochberg correction. Subgroup observations were reweighted toward the evaluated populations feature distributions for secondary comparisons of existing absolute SHAP values.
ResultsAmong the 1,857 included patients, 982 (52.9%) died during hospitalization. Random forest achieved the highest mean AUROC (0.925 {+/-} 0.010), followed by AdaBoost (0.919 {+/-} 0.017) and logistic regression (0.918 {+/-} 0.007), with variability reported as the SEM. Attribution profiles were more similar across repeated training trials of the same algorithm (mean Pearson r = 0.783; SEM, 0.004) than across algorithms within the same trial (mean Pearson r = 0.369; SEM, 0.006). The largest normalized attribution magnitudes were observed for dexamethasone use at home without oxygen support (6.186%; 95% CI, 5.795-6.577), PaO2/FiO2 ratio (3.014%; 95% CI, 2.787-3.242), shortness of breath (2.774%; 95% CI, 2.592-2.956), and FiO2 (2.560%; 95% CI, 2.284-2.836). Features differed in patient-level attribution alignment, indicating that importance magnitude and robustness provided complementary information. Among 1,763 eligible subgroup-feature comparisons, 64 had nominal one-sided p < 0.05, although none remained below 0.05 after Benjamini-Hochberg adjustment.
ConclusionsAlgorithms with similar predictive performance produced substantially different feature-attribution profiles. Normalized performance-weighted SHAP aggregation provided a representative relative-importance summary across algorithms, while resampling-based alignment assessment qualified the consistency of individual feature contributions. The resulting attribution patterns describe fitted-model behavior and should not be interpreted as causal clinical effects.
Related
- Ensemble machine learning for predicting breast cancer recurrence and mortality using clinical and hemogram data
- Interindividual variability in platelet aggregation among aspirin-treated patients: a large retrospective analysis
- Night-to-night sleep variability and all-cause and cardiovascular mortality: a prospective cohort study with a complementary in vitro circadian-disruption experiment
- Visit-to-Visit Blood Pressure Variability, Glycemic Status and the Risk of Dementia, Stroke, and All-Cause Mortality: A UK Biobank Cohort Study
- Clinical recognition, management, and research progress of infantile perianal protrusion: a narrative review and literature-based case aggregation
BibTeX
@misc{dai2026ensemble,
title = {{Ensemble SHAP Aggregation and Attribution Variability in Clinical Machine Learning: A COVID-19 Mortality Study}},
author = {N. Dai and R. K. Briceno and A. N. Castaneda and M. A. Tresierra and M. L. Esteban and M. E. Rosas and R. C. Hinojosa},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.07.26362459},
url = {https://www.medrxiv.org/content/10.64898/2026.09.07.26362459v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Ensemble SHAP Aggregation and Attribution Variability in Clinical Machine Learning: A COVID-19 Mortality Study
AU - N. Dai
AU - R. K. Briceno
AU - A. N. Castaneda
AU - M. A. Tresierra
AU - M. L. Esteban
AU - M. E. Rosas
AU - R. C. Hinojosa
PY - 2026
DA - 2026/09/07
PB - medRxiv
DO - 10.64898/2026.09.07.26362459
UR - https://www.medrxiv.org/content/10.64898/2026.09.07.26362459v1
AB - ObjectiveTo combine performance-weighted ensemble SHAP aggregation with resampling-based assessment of feature-importance variability and patient-level attribution alignment for interpreting COVID-19 mortality predictions. MethodsWe analyzed a prospective cohort of 1,857 patients hospitalized with COVID-19 at two hospitals in Peru. Ten predictive algorithms were evaluated using five-fold cross-validation, and their mean AUROC values determined their attribution-aggregation weights. Within each model and training trial, signed SHAP values were normalized by the mean total absolute attribution across evaluation patients before performance-weighted aggregation. Feature importance was summarized across 30 resampled training trials using means and normal-approximation 95% confidence intervals. Patient-level attribution alignment was assessed using cosine similarity between signed feature-attribution vectors for corresponding patients, with whole-trial patient-correspondence randomization and Benjamini-Hochberg correction. Subgroup observations were reweighted toward the evaluated populations feature distributions for secondary comparisons of existing absolute SHAP values. ResultsAmong the 1,857 included patients, 982 (52.9%) died during hospitalization. Random forest achieved the highest mean AUROC (0.925 {+/-} 0.010), followed by AdaBoost (0.919 {+/-} 0.017) and logistic regression (0.918 {+/-} 0.007), with variability reported as the SEM. Attribution profiles were more similar across repeated training trials of the same algorithm (mean Pearson r = 0.783; SEM, 0.004) than across algorithms within the same trial (mean Pearson r = 0.369; SEM, 0.006). The largest normalized attribution magnitudes were observed for dexamethasone use at home without oxygen support (6.186%; 95% CI, 5.795-6.577), PaO2/FiO2 ratio (3.014%; 95% CI, 2.787-3.242), shortness of breath (2.774%; 95% CI, 2.592-2.956), and FiO2 (2.560%; 95% CI, 2.284-2.836). Features differed in patient-level attribution alignment, indicating that importance magnitude and robustness provided complementary information. Among 1,763 eligible subgroup-feature comparisons, 64 had nominal one-sided p < 0.05, although none remained below 0.05 after Benjamini-Hochberg adjustment. ConclusionsAlgorithms with similar predictive performance produced substantially different feature-attribution profiles. Normalized performance-weighted SHAP aggregation provided a representative relative-importance summary across algorithms, while resampling-based alignment assessment qualified the consistency of individual feature contributions. The resulting attribution patterns describe fitted-model behavior and should not be interpreted as causal clinical effects.
ER -