Discordance between reported discrimination and translational-readiness evidence in AI-based ASCVD prediction: a systematic review with structured methodological audit
4 affiliations
- 1Beijing Friendship Hospital, Capital Medical University , ,
- 2Department of Physical Therapy, University of Illinois at Chicago , , ,
- 3John L Ochsner Heart and Vascular Institute , , ,
- 4Mayo Clinic , , ,
Abstract
Importance: The development of artificial intelligence (AI) models for atherosclerotic cardiovascular disease (ASCVD) risk prediction has rapidly accelerated. However, despite reports of high predictive accuracy, evidence supporting clinical implementation remains limited.
Objective: This study aimed to evaluate the methodological rigor of AI-based ASCVD prediction models and assess whether reported algorithmic discrimination is associated with prespecified reporting and translational-evidence gaps.
Data sources and study selection: A systematic search of PubMed, Embase, Web of Science, Scopus, and Cochrane CENTRAL was conducted for original studies published between 1 January 2020 and 27 March 2026. Studies reporting the development or validation of AI and machine learning prognostic models for adult ASCVD outcomes were included.
Data extraction and synthesis: Following the PRISMA guidelines, a structured methodological audit was performed. Five binary reporting and translational-evidence gaps were independently extracted: absent calibration assessment, lack of external validation, absent decision-analytic utility evaluation, undefined temporal prediction horizons, and lack of reproducibility or accessibility artifacts. A prespecified minimum evidence proxy for translational evaluation was defined.
Main outcomes and measures: The primary outcomes were the cumulative methodological gap burden (range, 0–5) and its association with the primary discrimination metric (area under the receiver operating characteristic curve [AUC] or C-index).
Results: Among 101 primary models evaluated, the median discrimination performance was 0.790 (IQR, 0.734–0.833). However, a substantial cumulative gap burden was observed, with only 6.9% of the models meeting the prespecified translational-readiness proxy. Reported discrimination was not associated with methodological completeness; top-quartile models (AUC ≥ 0.833) exhibited high gap burdens at rates comparable to lower-performing models. Reporting and translational-evidence gaps systematically co-occurred, predominantly driven by the simultaneous absence of external validation, calibration assessment, and reproducibility or accessibility artifacts. The use of advanced algorithmic architectures (e.g., deep learning) and multimodal data integration did not independently reduce the reporting gap burden.
Conclusion and relevance: In AI-based ASCVD risk prediction, reported algorithmic discrimination is a poor surrogate for clinical readiness. The literature features models with high discrimination but critical methodological deficits. These findings suggest that evaluation frameworks should consider incorporating calibration, external validation, and formal decision-analytic utility assessment alongside discrimination metrics.
Related
- Integrated CDK4/6 inhibitor resistance and primary-metastatic discordance in hormone receptor-positive/HER2-negative metastatic breast cancer: a translational framework focused on invasive ductal carcinoma with gastric metastasis.
- Animal models of anemia of inflammation: Methodological advances, limitations, and translational applications
- Artificial intelligence for women’s reproductive health: A scoping review of global diagnostic trends, methodological gaps, and a translational research agenda for low-resource settings
- Speech markers of depression: cross-sectional discrimination, longitudinal association, and treatment response prediction
- Molecular and Conventional Tests in Onychomycosis: A Systematic Review of Paired Diagnostic Discordance and Clinical Interpretation
BibTeX
@article{liu2026discordance,
title = {{Discordance between reported discrimination and translational-readiness evidence in AI-based ASCVD prediction: a systematic review with structured methodological audit}},
author = {Ruifeng Liu and Ross Arena and Carl Lavie and Adelaide M. Arruda-Olson and Dejana Popovic},
journal = {Frontiers in public health},
year = {2026},
month = sep,
doi = {10.3389/fpubh.2026.1963871},
url = {https://europepmc.org/article/PMC/PMC13662102},
note = {CC BY},
}
RIS
TY - JOUR
TI - Discordance between reported discrimination and translational-readiness evidence in AI-based ASCVD prediction: a systematic review with structured methodological audit
AU - Ruifeng Liu
AU - Ross Arena
AU - Carl Lavie
AU - Adelaide M. Arruda-Olson
AU - Dejana Popovic
PY - 2026
DA - 2026/09/30
JO - Frontiers in public health
DO - 10.3389/fpubh.2026.1963871
UR - https://europepmc.org/article/PMC/PMC13662102
AB - Importance: The development of artificial intelligence (AI) models for atherosclerotic cardiovascular disease (ASCVD) risk prediction has rapidly accelerated. However, despite reports of high predictive accuracy, evidence supporting clinical implementation remains limited. Objective: This study aimed to evaluate the methodological rigor of AI-based ASCVD prediction models and assess whether reported algorithmic discrimination is associated with prespecified reporting and translational-evidence gaps. Data sources and study selection: A systematic search of PubMed, Embase, Web of Science, Scopus, and Cochrane CENTRAL was conducted for original studies published between 1 January 2020 and 27 March 2026. Studies reporting the development or validation of AI and machine learning prognostic models for adult ASCVD outcomes were included. Data extraction and synthesis: Following the PRISMA guidelines, a structured methodological audit was performed. Five binary reporting and translational-evidence gaps were independently extracted: absent calibration assessment, lack of external validation, absent decision-analytic utility evaluation, undefined temporal prediction horizons, and lack of reproducibility or accessibility artifacts. A prespecified minimum evidence proxy for translational evaluation was defined. Main outcomes and measures: The primary outcomes were the cumulative methodological gap burden (range, 0–5) and its association with the primary discrimination metric (area under the receiver operating characteristic curve [AUC] or C-index). Results: Among 101 primary models evaluated, the median discrimination performance was 0.790 (IQR, 0.734–0.833). However, a substantial cumulative gap burden was observed, with only 6.9% of the models meeting the prespecified translational-readiness proxy. Reported discrimination was not associated with methodological completeness; top-quartile models (AUC ≥ 0.833) exhibited high gap burdens at rates comparable to lower-performing models. Reporting and translational-evidence gaps systematically co-occurred, predominantly driven by the simultaneous absence of external validation, calibration assessment, and reproducibility or accessibility artifacts. The use of advanced algorithmic architectures (e.g., deep learning) and multimodal data integration did not independently reduce the reporting gap burden. Conclusion and relevance: In AI-based ASCVD risk prediction, reported algorithmic discrimination is a poor surrogate for clinical readiness. The literature features models with high discrimination but critical methodological deficits. These findings suggest that evaluation frameworks should consider incorporating calibration, external validation, and formal decision-analytic utility assessment alongside discrimination metrics.
SN - 2296-2565
ER -