Skip to content
DraconiticResearch
Search Sources API
226,742 studies31 sourcesUpdated 36 min ago
Peer-reviewedCC BY 4.0Frontiers in Cardiovascular Medicine

Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset

  1. 1Graduate School of Heilongjiang University of Traditional Chinese Medicine
  2. 2Department of Cardiology III, First Affiliated Hospital of Heilongjiang University of Traditional Chinese Medicine

Abstract

Background: The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance.

Objective: To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations.

Methods: Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed.

Results: Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance.

Conclusions: Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.

Related

  1. On-Device ECG and Vital-Sign Prediction of 24-Hour Mortality and Unplanned ICU Admission in Emergency Department Patients: A Retrospective Evaluation of MEDI-AI
    Diagnostics (Basel, Switzerland)Peer-reviewedCC BYFull textGeneral medicineEurope PMC
  2. Competing Risk Prediction of COVID-19 Hospital Mortality with Model Diagnostics and Exploratory Evaluation in Mexico
    Zahra Sabzevari, Hamid Reza Marateb, Reza Rasti +3
    Diagnostics (Basel, Switzerland)Peer-reviewedCC BYFull textGeneral medicineEurope PMC
  3. Temporal interpretation of physiologic group contributions for ICU mortality prediction
    Jinwoong Kim, Yeeun Kim, Sangjin Park
    Scientific ReportsPeer-reviewedCC BYFull textGeneral medicineDOAJ
  4. Temporal Validation and Simplification of Mortality Prediction for Perioperative Cardiopulmonary Resuscitation
    L. Chen, S. A. Justice, S. Muthupalaniappan +2
    medRxivPreprintCC BY-NC-NDFull textSurgery & anaesthesiamedRxiv
  5. Incremental value of bilateral ankle-brachial index for ten-year heart and cerebrovascular mortality prediction: an exploratory temporal benchmark in NHANES
    L. You, Y. Guo, X. Zhong +4
    medRxivPreprintCC BYFull textCardiovascular medicinemedRxiv
BibTeX
@article{diao2026temporal,
  title = {{Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset}},
  author = {Yinuo Diao and Kuikui Zhang and Wenzeng Wang},
  journal = {Frontiers in Cardiovascular Medicine},
  year = {2026},
  month = nov,
  doi = {10.3389/fcvm.2026.1980349},
  url = {https://www.frontiersin.org/articles/10.3389/fcvm.2026.1980349/full},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Temporal leakage and evaluation-boundary effects in heart failure mortality prediction: a unified reanalysis of the 299-patient heart failure clinical records dataset
AU  - Yinuo Diao
AU  - Kuikui Zhang
AU  - Wenzeng Wang
PY  - 2026
DA  - 2026/11/16
JO  - Frontiers in Cardiovascular Medicine
DO  - 10.3389/fcvm.2026.1980349
UR  - https://www.frontiersin.org/articles/10.3389/fcvm.2026.1980349/full
AB  - Background: The 299-patient Heart Failure Clinical Records dataset is widely used for mortality prediction, yet reported performance varies substantially. Differences in prediction time, follow-up handling, resampling, and data splitting may materially affect apparent model performance.  Objective: To examine how prediction-time alignment, modelling procedures, and evaluation boundaries influence performance estimates on this shared benchmark, using a targeted methodological audit and harmonized reanalysis of representative modelling configurations.  Methods: Four representative modelling routes were evaluated using the same dataset. Admission-time classification used 11 baseline variables and stratified 10-fold cross-validation, with applicable resampling and feature-construction procedures restricted to the training data. A parsimonious random forest using ejection fraction and serum creatinine was also assessed. Follow-up duration was excluded from baseline classification and retained for time-to-event analyses. Sensitivity analyses examined follow-up-duration inclusion and changes in preprocessing boundaries. Patient-level bootstrap confidence intervals, repeated-seed analyses, calibration, and exploratory decision-curve analysis were additionally performed.  Results: Fold-mean AUROCs across the four 11-variable classification configurations ranged from 0.724 to 0.784. The two-feature random forest achieved a pooled out-of-fold AUROC of 0.779 (95% CI, 0.721–0.832), with a paired difference of 0.007 (95% CI, −0.047 to 0.062) from the 11-variable random-forest comparator. Adding follow-up duration increased fold-mean AUROC by 0.121–0.135 within matched predictor sets. The two-feature model retained substantial discrimination but showed limited calibration, with a calibration slope of 0.234 (95% CI, 0.154–0.404). Preprocessing sensitivity analyses also produced substantial variation in apparent performance.  Conclusions: Prediction-time alignment and evaluation procedures materially influenced performance estimates on this benchmark. Similar discrimination did not imply comparable probability calibration or equivalent patient-level risk estimates. Clinical interpretation therefore requires explicit definition of prediction time, outcome horizon, model implementation, and evaluation procedures, followed by validation in independent contemporary cohorts.
SN  - 2297-055X
ER  -