An interpretable machine learning model for predicting intensive care unit mortality risk in elderly patients with concomitant cardiogenic shock and sepsis: Development and external validation using two large critical care databases
Abstract
Introduction Older adults with concomitant cardiogenic shock (CS) and sepsis represent a high-risk mixed-shock population in whom early prognosis is difficult. We aimed to develop and externally validate an interpretable machine-learning model for ICU mortality. Methods Patients aged ≥65 years with concomitant CS and sepsis were identified from MIMIC-IV v3.1 (n=1,219) and eICU-CRD v2.0 (n=166). MIMIC-IV was stratified 70:30 into training (n=853) and internal validation (n=366), while eICU-CRD was reserved for external validation. Feature selection used the intersection of Boruta and LASSO in the training cohort. Ten models were tuned by stratified 5-fold cross-validation; the primary model family was selected by the highest mean training cross-validated AUROC before validation. Performance assessment included bootstrap 95% confidence intervals, calibration, decision curve analysis, and SHAP. Results Nine predictors were retained: AKI, T2DM, respiratory rate, lactate, creatinine, BUN, ALT, SOFA, and SAPS II. Stacking ranked first by the training-CV model-selection rule (mean CV AUROC=0.690). AUROC was 0.742 (95% CI 0.690-0.791) in internal validation and 0.633 (0.546-0.718) in external validation. At the locked threshold, external sensitivity was 0.750 and specificity 0.441. The external Brier score of 0.287 exceeded the prevalence-only null Brier score of 0.237 (scaled Brier=-0.212). SOFA alone showed similar external discrimination (AUROC=0.646) with a lower Brier score (0.226), indicating no clear external advantage of the ensemble over simpler benchmarks. Cross-database standardized mean differences demonstrated substantial distribution shift. Conclusions The training-selected Stacking model showed moderate internal discrimination but limited external discrimination and probabilistic calibration, without a clear external advantage over simpler benchmarks. The model should be considered investigational pending prospective evaluation and site-specific validation or recalibration.
Related
- Development and validation of an interpretable machine learning model for predicting in-hospital mortality among critically ill patients with liver cirrhosis
- Development and Validation of an Interpretable Machine‐Learning Model for Predicting Suboptimal Clinical Response After Sleeve Gastrectomy: A Multicenter Study
- Development and validation of interpretable machine learning models for predicting cumulative live birth rates in assisted reproductive technology: a SHAP-based analysis.
- Development and internal validation of an interpretable machine learning model for predicting vancomycin-induced nephrotoxicity in hospitalized children
- Development and validation of an interpretable machine learning model for predicting intravenous contrast media extravasation risk in CT imaging
BibTeX
@article{chen2026interpretable,
title = {{An interpretable machine learning model for predicting intensive care unit mortality risk in elderly patients with concomitant cardiogenic shock and sepsis: Development and external validation using two large critical care databases}},
author = {Han Chen and Yuming Wu and Pu Chen and Zijian Lin and Qingjuan Zheng and Ling Wang},
journal = {Digital Health},
year = {2026},
month = oct,
doi = {10.1177/20552076261501511},
url = {https://doi.org/10.1177/20552076261501511},
note = {CC BY},
}
RIS
TY - JOUR
TI - An interpretable machine learning model for predicting intensive care unit mortality risk in elderly patients with concomitant cardiogenic shock and sepsis: Development and external validation using two large critical care databases
AU - Han Chen
AU - Yuming Wu
AU - Pu Chen
AU - Zijian Lin
AU - Qingjuan Zheng
AU - Ling Wang
PY - 2026
DA - 2026/10/23
JO - Digital Health
DO - 10.1177/20552076261501511
UR - https://doi.org/10.1177/20552076261501511
AB - Introduction Older adults with concomitant cardiogenic shock (CS) and sepsis represent a high-risk mixed-shock population in whom early prognosis is difficult. We aimed to develop and externally validate an interpretable machine-learning model for ICU mortality. Methods Patients aged ≥65 years with concomitant CS and sepsis were identified from MIMIC-IV v3.1 (n=1,219) and eICU-CRD v2.0 (n=166). MIMIC-IV was stratified 70:30 into training (n=853) and internal validation (n=366), while eICU-CRD was reserved for external validation. Feature selection used the intersection of Boruta and LASSO in the training cohort. Ten models were tuned by stratified 5-fold cross-validation; the primary model family was selected by the highest mean training cross-validated AUROC before validation. Performance assessment included bootstrap 95% confidence intervals, calibration, decision curve analysis, and SHAP. Results Nine predictors were retained: AKI, T2DM, respiratory rate, lactate, creatinine, BUN, ALT, SOFA, and SAPS II. Stacking ranked first by the training-CV model-selection rule (mean CV AUROC=0.690). AUROC was 0.742 (95% CI 0.690-0.791) in internal validation and 0.633 (0.546-0.718) in external validation. At the locked threshold, external sensitivity was 0.750 and specificity 0.441. The external Brier score of 0.287 exceeded the prevalence-only null Brier score of 0.237 (scaled Brier=-0.212). SOFA alone showed similar external discrimination (AUROC=0.646) with a lower Brier score (0.226), indicating no clear external advantage of the ensemble over simpler benchmarks. Cross-database standardized mean differences demonstrated substantial distribution shift. Conclusions The training-selected Stacking model showed moderate internal discrimination but limited external discrimination and probabilistic calibration, without a clear external advantage over simpler benchmarks. The model should be considered investigational pending prospective evaluation and site-specific validation or recalibration.
SN - 2055-2076
ER -