Development and external evaluation of an interpretable machine-learning model for early prediction of organ failure in higher-risk acute pancreatitis patients: A multicentre cohort study
10 affiliations
- 1Pancreas Center, First Affiliated Hospital with Nanjing Medical University, Nanjing, Jiangsu, China
- 2Department of Geriatric Gastroenterology, Xiangya Hospital, Central South University, Changsha, Hunan, China
- 3Institute of Systems, Molecular and Integrative Biology, University of Liverpool, Liverpool, United Kingdom
- 4Emergency Department, the Third Xiangya Hospital, Central South University, Changsha, Hunan, China
- 5West China Centre of Excellence for Pancreatitis, Institute of Integrated Traditional Chinese and Western Medicine, West China-Liverpool Biomedical Research Centre, Chengdu, Sichuan, China
- 6Pediatric Hematology Laboratory, Division of Hematology/Oncology, Department of Pediatrics, The Seventh Affiliated Hospital of Sun Yat-Sen University, Shenzhen, Guangdong, China
- 7Department of Ophthalmology, Nanfang Hospital, Southern Medical University, Guangzhou, Guangdong, China
- 8Department of Information Center, The Affiliated Jiangning Hospital of Nanjing Medical University, Nanjing, Jiangsu, China
- 9Computational Biology Facility, University of Liverpool, Liverpool, United Kingdom
- 10Department of Radiology, First Affiliated Hospital with Nanjing Medical University, Nanjing Medical University, Nanjing, Jiangsu, China
Abstract
Organ failure (OF) is the most important determinant of prognosis in acute pancreatitis (AP) and its duration defines disease severity. Early identification of individuals at high risk of developing OF is crucial. We aimed to develop machine-learning models, benchmarked against a feedforward multilayer-perceptron (MLP) neural network, to predict new-onset OF at admission using static admission-day variables. In this study, data were extracted from MIMIC-IV (development cohort) and a multicentre AP cohort from two Chinese tertiary teaching hospitals (evaluation cohort), which excluded mild AP and therefore comprised patients requiring ICU-level or high-dependency care. Features were selected with Boruta and LASSO. Five machine-learning models and one multilayer-perceptron benchmark were developed in MIMIC-IV and externally evaluated in the Xiangya cohort without updating. SHapley Additive exPlanations (SHAP) were used to visualize decision-making patterns and individual prediction interpretations. The best-performing model was deployed as an interactive, web-based tool. We found that in the discovery cohort, 341 (23.9%) of 1429 AP patients developed new-onset OF within 28 days of admission while 45 of 216 patients (20.8%) in the validation cohort developed OF. Boruta and LASSO algorithms identified six key predictors including blood urea nitrogen, platelets, triglyceride-glucose index, albumin, white blood cells, and partial thromboplastin time, which were used to construct the ML and MLP models. XGBoost gave the best discrimination on external evaluation (AUC 0.837, 95% CI 0.771–0.902). Logistic recalibration improved calibration, and the recalibrated XGBoost model was implemented as a web-based research prototype (Decent app). In conclusion, XGBoost predicted new-onset OF with good discrimination in a severity-enriched AP cohort, and SHAP made individual predictions interpretable. Model selection, threshold selection and recalibration all used the evaluation cohort, so the deployed model still requires an independent cohort. Prospective usability and clinical-impact studies are needed before clinical use. Author summary: Acute pancreatitis (AP) is a serious inflammatory condition where the development of organ failure (OF) is the most critical factor determining patient outcomes. Early identification of patients at high risk for OF is essential for improving care, but accurate prediction at the time of hospital admission remains a challenge. In this study, we developed machine-learning models, benchmarked against a multilayer perceptron, and evaluated them externally to predict new-onset OF in AP patients on admission to hospital. Using data from large international cohorts, we identified six key clinical characteristics at admission, including blood urea nitrogen, platelets, triglyceride-glucose index, albumin, white blood cells, and partial thromboplastin time levels, to train our models. Among the algorithms tested on admission-day tabular data, XGBoost performed best. The cohort used for evaluation excluded mild AP, so our results apply to patients needing ICU-level, high-dependency or close monitoring, not to unselected admissions. We then recalibrated the XGBoost model in the same cohort and built a web application (Decent app) that returns a risk estimate with an explanation of the factors behind it. Because the recalibration and the risk threshold came from the cohort in which we assessed them, the application has not been independently validated. It is a research prototype for risk calculation rather than a decision-support tool, and prospective usability, workflow and clinical-impact studies are needed first.
Related
- Evaluation of clinical prediction models: Decomposing performance uncertainty into model development and validation components
- What Comes Next and Why: Interpretable Next-Event Prediction with Neuro-Symbolic Rules
- BRP-GABLE: An Interpretable Rule-Based Prediction Framework for Multiple Outcome Types
- Fragment-Level Graph Encoding for Interpretable Molecular Property Prediction
- Interpretable Social Network Churn Prediction via Ergodic Markov Chains
BibTeX
@article{wu2026development,
title = {{Development and external evaluation of an interpretable machine-learning model for early prediction of organ failure in higher-risk acute pancreatitis patients: A multicentre cohort study}},
author = {Di Wu and Wenhao Cai and Chunmei Chen and Minting Chen and Yang Lv and Yilin Huang and Anthony Evans and Juan Lin and Diane Latawiec and Arjun Kattakayam and Rajarshi Mukherjee and Wei Huang and Qing Xia and Jie Xiao and Chunqiu Su and Jie Peng and Kuirong Jiang and Robert Sutton},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001735},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001735},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Development and external evaluation of an interpretable machine-learning model for early prediction of organ failure in higher-risk acute pancreatitis patients: A multicentre cohort study
AU - Di Wu
AU - Wenhao Cai
AU - Chunmei Chen
AU - Minting Chen
AU - Yang Lv
AU - Yilin Huang
AU - Anthony Evans
AU - Juan Lin
AU - Diane Latawiec
AU - Arjun Kattakayam
AU - Rajarshi Mukherjee
AU - Wei Huang
AU - Qing Xia
AU - Jie Xiao
AU - Chunqiu Su
AU - Jie Peng
AU - Kuirong Jiang
AU - Robert Sutton
PY - 2026
DA - 2026/09/22
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001735
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001735
AB - Organ failure (OF) is the most important determinant of prognosis in acute pancreatitis (AP) and its duration defines disease severity. Early identification of individuals at high risk of developing OF is crucial. We aimed to develop machine-learning models, benchmarked against a feedforward multilayer-perceptron (MLP) neural network, to predict new-onset OF at admission using static admission-day variables. In this study, data were extracted from MIMIC-IV (development cohort) and a multicentre AP cohort from two Chinese tertiary teaching hospitals (evaluation cohort), which excluded mild AP and therefore comprised patients requiring ICU-level or high-dependency care. Features were selected with Boruta and LASSO. Five machine-learning models and one multilayer-perceptron benchmark were developed in MIMIC-IV and externally evaluated in the Xiangya cohort without updating. SHapley Additive exPlanations (SHAP) were used to visualize decision-making patterns and individual prediction interpretations. The best-performing model was deployed as an interactive, web-based tool. We found that in the discovery cohort, 341 (23.9%) of 1429 AP patients developed new-onset OF within 28 days of admission while 45 of 216 patients (20.8%) in the validation cohort developed OF. Boruta and LASSO algorithms identified six key predictors including blood urea nitrogen, platelets, triglyceride-glucose index, albumin, white blood cells, and partial thromboplastin time, which were used to construct the ML and MLP models. XGBoost gave the best discrimination on external evaluation (AUC 0.837, 95% CI 0.771–0.902). Logistic recalibration improved calibration, and the recalibrated XGBoost model was implemented as a web-based research prototype (Decent app). In conclusion, XGBoost predicted new-onset OF with good discrimination in a severity-enriched AP cohort, and SHAP made individual predictions interpretable. Model selection, threshold selection and recalibration all used the evaluation cohort, so the deployed model still requires an independent cohort. Prospective usability and clinical-impact studies are needed before clinical use. Author summary: Acute pancreatitis (AP) is a serious inflammatory condition where the development of organ failure (OF) is the most critical factor determining patient outcomes. Early identification of patients at high risk for OF is essential for improving care, but accurate prediction at the time of hospital admission remains a challenge. In this study, we developed machine-learning models, benchmarked against a multilayer perceptron, and evaluated them externally to predict new-onset OF in AP patients on admission to hospital. Using data from large international cohorts, we identified six key clinical characteristics at admission, including blood urea nitrogen, platelets, triglyceride-glucose index, albumin, white blood cells, and partial thromboplastin time levels, to train our models. Among the algorithms tested on admission-day tabular data, XGBoost performed best. The cohort used for evaluation excluded mild AP, so our results apply to patients needing ICU-level, high-dependency or close monitoring, not to unselected admissions. We then recalibrated the XGBoost model in the same cohort and built a web application (Decent app) that returns a risk estimate with an explanation of the factors behind it. Because the recalibration and the risk threshold came from the cohort in which we assessed them, the application has not been independently validated. It is a research prototype for risk calculation rather than a decision-support tool, and prospective usability, workflow and clinical-impact studies are needed first.
SN - 2767-3170
ER -