CardioCLIP: sequential multimodal prediction of echocardiographic abnormalities and natriuretic peptide elevation from electrocardiogram and chest radiographs.
5 affiliations
- 1Kobe University School of Medicine, Kobe, Japan.
- 2Division of Molecular Epidemiology, Kobe University Graduate School of Medicine, 7-5-2, Kusunoki-cho, Chuo-ku, Kobe 650-0017, Japan.
- 3Division of Cardiovascular Medicine, Department of Internal Medicine, Kobe University Hospital, Kobe, Japan.
- 4Division of Medical Informatics, Department of Internal Related, Kobe University Graduate School of Medicine, Kobe, Japan.
- 5Faculty of Data Science, Shiga University, Shiga, Japan.
Abstract
Aims
Diagnostic reasoning in clinical practice is inherently sequential and multimodal, with physicians integrating heterogeneous examinations under constraints such as missing data and variable test order. However, most multimodal machine learning models assume simultaneous availability of all modalities and fail to reflect this workflow.Methods and results
We propose CardioCLIP, a sequential multimodal framework that embeds electrocardiograms (ECGs) and chest X-rays (CXRs) into a shared pathological representation space. The model consists of (i) a self-supervised embedding stage aligning ECG-CXR pairs from the same patient and day, and (2) a gated recurrent unit-based prediction stage that estimates clinical indicators as data are sequentially observed. The model was trained on 9458 patients from a single tertiary centre and evaluated for B-type natriuretic peptide (BNP), left ventricular ejection fraction (LVEF), and E/e'. External validation was conducted using MIMIC-IV.CardioCLIP achieved strong cross-modal alignment and improved discrimination for selected echocardiographic abnormalities and natriuretic peptide elevation (AUC: 0.90 for BNP > 100 pg/mL, 0.87 for LVEF < 50%, and 0.80 for E/e' > 15). It also showed discrimination for subsequent severe BNP elevation (BNP > 400 pg/mL within 3 years) with an AUC of 0.79. External validation in MIMIC-IV showed an AUC of 0.89 for N-terminal pro-BNP elevation, supporting potential transportability for this endpoint.Conclusion
CardioCLIP demonstrated the feasibility of order-flexible sequential multimodal prediction. By leveraging routinely acquired ECG and CXR data, it may provide a non-invasive adjunct for identifying echocardiographic abnormalities and natriuretic peptide elevation in cardiovascular care, subject to prospective clinical validation.
Related
- Long-term myocardial deformation abnormalities and biomarker associations in pediatric myopericarditis: a speckle-tracking echocardiographic study
- Sacubitrilat exerts potent antiarrhythmic effects through activation of natriuretic peptide receptor-C
- Serial N-Terminal pro B-Type Natriuretic Peptide in the Community ― Dynamic Risk Signal or Incremental Predictor? ―
- Natriuretic Peptide Receptor-C (NPR3) in Pulmonary Hypertension Associated with Chronic Lung Disease: A PRISMA-ScR Scoping Review of Diagnostic, Prognostic, Mechanistic, and Therapeutic Evidence
- Multi-target effects and clinical optimization treatment strategies of recombinant human B-type natriuretic peptide in acute heart failure and myocardial infarction
BibTeX
@article{shibata2026cardioclip,
title = {{CardioCLIP: sequential multimodal prediction of echocardiographic abnormalities and natriuretic peptide elevation from electrocardiogram and chest radiographs.}},
author = {Satoki Shibata and Makoto Nishimori and Yu Nishihara and Eiichi Maeda and Masaaki Iiyama and Masakazu Shinohara},
journal = {European heart journal. Digital health},
year = {2026},
month = oct,
doi = {10.1093/ehjdh/ztag158},
url = {https://europepmc.org/article/MED/42927257},
note = {CC BY},
}
RIS
TY - JOUR
TI - CardioCLIP: sequential multimodal prediction of echocardiographic abnormalities and natriuretic peptide elevation from electrocardiogram and chest radiographs.
AU - Satoki Shibata
AU - Makoto Nishimori
AU - Yu Nishihara
AU - Eiichi Maeda
AU - Masaaki Iiyama
AU - Masakazu Shinohara
PY - 2026
DA - 2026/10/01
JO - European heart journal. Digital health
DO - 10.1093/ehjdh/ztag158
UR - https://europepmc.org/article/MED/42927257
AB - Aims Diagnostic reasoning in clinical practice is inherently sequential and multimodal, with physicians integrating heterogeneous examinations under constraints such as missing data and variable test order. However, most multimodal machine learning models assume simultaneous availability of all modalities and fail to reflect this workflow.Methods and results We propose CardioCLIP, a sequential multimodal framework that embeds electrocardiograms (ECGs) and chest X-rays (CXRs) into a shared pathological representation space. The model consists of (i) a self-supervised embedding stage aligning ECG-CXR pairs from the same patient and day, and (2) a gated recurrent unit-based prediction stage that estimates clinical indicators as data are sequentially observed. The model was trained on 9458 patients from a single tertiary centre and evaluated for B-type natriuretic peptide (BNP), left ventricular ejection fraction (LVEF), and E/e'. External validation was conducted using MIMIC-IV.CardioCLIP achieved strong cross-modal alignment and improved discrimination for selected echocardiographic abnormalities and natriuretic peptide elevation (AUC: 0.90 for BNP > 100 pg/mL, 0.87 for LVEF < 50%, and 0.80 for E/e' > 15). It also showed discrimination for subsequent severe BNP elevation (BNP > 400 pg/mL within 3 years) with an AUC of 0.79. External validation in MIMIC-IV showed an AUC of 0.89 for N-terminal pro-BNP elevation, supporting potential transportability for this endpoint.Conclusion CardioCLIP demonstrated the feasibility of order-flexible sequential multimodal prediction. By leveraging routinely acquired ECG and CXR data, it may provide a non-invasive adjunct for identifying echocardiographic abnormalities and natriuretic peptide elevation in cardiovascular care, subject to prospective clinical validation.
SN - 2634-3916
ER -