Comparison of Internal Validation Strategies for Estimating Machine Learning Model Performance in One-Year Mortality Prediction in Hepatocellular Carcinoma
- 1Department of Biostatistics and Medical Informatics, Faculty of Medicine, Inonu University, 44280 Malatya, Türkiye
- 2Department of Surgery and Liver Transplantation, Faculty of Medicine, Inonu University, 44280 Malatya, Türkiye
Abstract
Background: Machine learning (ML) performance estimates in hepatocellular carcinoma (HCC) may depend on internal validation strategy, especially in small datasets. Aim: To develop models for one-year mortality prediction and compare performance, calibration, decision-analytic performance, and uncertainty estimates using a conventional train–test split and nested cross-validation (nCV). Methods: We analyzed a public dataset of 165 patients with HCC, including 63 one-year deaths. Random Forest (RF), XGBoost, and CatBoost were developed using fold-specific preprocessing, cost-sensitive learning, Optuna-based hyperparameter optimization, and training-based threshold selection. L2-penalized logistic regression provided a conventional benchmark. Models were evaluated using a stratified 80/20 split and five-fold nCV with five inner folds. Six stratified splits assessed partition variability, and unweighted models assessed the effect of class weighting. Discrimination, threshold-dependent classification, calibration, and decision-analytic performance were assessed using ROC-AUC, PR-AUC, balanced accuracy, Matthew’s correlation coefficient, Brier score, calibration curves, and decision-curve analysis. Confidence intervals used 2000 bootstrap resamples; SHAP was used for interpretation. Results: In the conventional split, RF and CatBoost achieved the highest test-set ROC-AUC (0.892 each), while RF achieved the highest PR-AUC (0.860). Under nCV, RF had the highest mean outer-fold ROC-AUC (0.788) and pooled out-of-fold ROC-AUC (0.784). XGBoost had the highest mean PR-AUC (0.673), whereas CatBoost had the highest pooled PR-AUC (0.648). RF had the lowest mean Brier score (0.188), followed by XGBoost (0.194) and CatBoost (0.203). Calibration varied across models, with pooled calibration slopes of 1.438, 0.691, and 0.427, respectively. Across six 80/20 splits, mean ROC-AUC values were 0.792, 0.769, and 0.761 for RF, XGBoost, and CatBoost, respectively, with substantial partition-dependent variation. Removing class weighting yielded broadly comparable performance. Bootstrap confidence intervals were narrower for pooled nCV predictions, but the evaluation samples differed in size (165 versus 33 patients), precluding attribution of this difference to validation strategy alone. Decision-curve analysis was exploratory because no clinically actionable threshold was prespecified. SHAP identified AFP, ALP, hemoglobin, albumin, performance status, and AST among the most influential RF predictors; the highest-ranked predictors were generally consistent across outer folds. Conclusions: nCV separated model development from evaluation and reduced reliance on a single small test set. RF was selected by the prespecified mean outer-fold ROC-AUC criterion, but differences among tree-based models were small and no clear discrimination advantage over penalized logistic regression was demonstrated. External validation is needed before clinical use.
Related
- Development and internal validation of a nomogram for estimating concurrent superficial vein thrombosis at presentation in patients with lower extremity varicose veins
- Head-to-head comparison of [18F]FAPI and [18F]FDG PET/CT in the clinical value of hepatocellular carcinoma
- Comparison of predictive models for estimating scorpion, snake, and spider envenomation under warming scenarios in São Paulo State, Brazil
- Estimating heart rate variability using facial video photoplethysmography: a pilot validation study
- Navigating HPV vaccination challenges in low-resource settings: Practical strategies for estimating the size of out-of-school girls
BibTeX
@article{kucukakcali2026comparison,
title = {{Comparison of Internal Validation Strategies for Estimating Machine Learning Model Performance in One-Year Mortality Prediction in Hepatocellular Carcinoma}},
author = {Zeynep Kucukakcali and Sami Akbulut and Kemal Baris Sarici},
journal = {Journal of Clinical Medicine},
year = {2026},
month = oct,
doi = {10.3390/jcm15207899},
url = {https://www.mdpi.com/2077-0383/15/20/7899},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Comparison of Internal Validation Strategies for Estimating Machine Learning Model Performance in One-Year Mortality Prediction in Hepatocellular Carcinoma
AU - Zeynep Kucukakcali
AU - Sami Akbulut
AU - Kemal Baris Sarici
PY - 2026
DA - 2026/10/01
JO - Journal of Clinical Medicine
DO - 10.3390/jcm15207899
UR - https://www.mdpi.com/2077-0383/15/20/7899
AB - Background: Machine learning (ML) performance estimates in hepatocellular carcinoma (HCC) may depend on internal validation strategy, especially in small datasets. Aim: To develop models for one-year mortality prediction and compare performance, calibration, decision-analytic performance, and uncertainty estimates using a conventional train–test split and nested cross-validation (nCV). Methods: We analyzed a public dataset of 165 patients with HCC, including 63 one-year deaths. Random Forest (RF), XGBoost, and CatBoost were developed using fold-specific preprocessing, cost-sensitive learning, Optuna-based hyperparameter optimization, and training-based threshold selection. L2-penalized logistic regression provided a conventional benchmark. Models were evaluated using a stratified 80/20 split and five-fold nCV with five inner folds. Six stratified splits assessed partition variability, and unweighted models assessed the effect of class weighting. Discrimination, threshold-dependent classification, calibration, and decision-analytic performance were assessed using ROC-AUC, PR-AUC, balanced accuracy, Matthew’s correlation coefficient, Brier score, calibration curves, and decision-curve analysis. Confidence intervals used 2000 bootstrap resamples; SHAP was used for interpretation. Results: In the conventional split, RF and CatBoost achieved the highest test-set ROC-AUC (0.892 each), while RF achieved the highest PR-AUC (0.860). Under nCV, RF had the highest mean outer-fold ROC-AUC (0.788) and pooled out-of-fold ROC-AUC (0.784). XGBoost had the highest mean PR-AUC (0.673), whereas CatBoost had the highest pooled PR-AUC (0.648). RF had the lowest mean Brier score (0.188), followed by XGBoost (0.194) and CatBoost (0.203). Calibration varied across models, with pooled calibration slopes of 1.438, 0.691, and 0.427, respectively. Across six 80/20 splits, mean ROC-AUC values were 0.792, 0.769, and 0.761 for RF, XGBoost, and CatBoost, respectively, with substantial partition-dependent variation. Removing class weighting yielded broadly comparable performance. Bootstrap confidence intervals were narrower for pooled nCV predictions, but the evaluation samples differed in size (165 versus 33 patients), precluding attribution of this difference to validation strategy alone. Decision-curve analysis was exploratory because no clinically actionable threshold was prespecified. SHAP identified AFP, ALP, hemoglobin, albumin, performance status, and AST among the most influential RF predictors; the highest-ranked predictors were generally consistent across outer folds. Conclusions: nCV separated model development from evaluation and reduced reliance on a single small test set. RF was selected by the prespecified mean outer-fold ROC-AUC criterion, but differences among tree-based models were small and no clear discrimination advantage over penalized logistic regression was demonstrated. External validation is needed before clinical use.
SN - 2077-0383
ER -