Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms
11 affiliations
- 1National Heart and Lung Institute, Imperial College London, London, United Kingdom
- 2Department of Cardiology, Chelsea and Westminster Hospital NHS Foundation Trust, London, United Kingdom
- 3Department of Cardiology, Zhongshan Hospital of Fudan University, Shanghai Institute of Cardiovascular Diseases, National Clinical Research Centre for Interventional Medicine, Shanghai, China
- 4Department of Information Technology, Uppsala University, Uppsala, Sweden
- 5Vanderbilt Institute for Clinical and Translational Research, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America
- 6Center for Digital Genomic Medicine, Department of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America
- 7Division of Cardiovascular Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, United States of America
- 8Department of Internal Medicine, Faculdade de Medicina, and Telehealth Center and Cardiology Service, Hospital das Clínicas, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil
- 9Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, United States of America
- 10Harvard-Thorndike Electrophysiology Institute, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, United States of America
- 11Department of Cardiology, Imperial College Healthcare NHS Trust, London, United Kingdom
Abstract
Contrastive learning is a widely adopted self-supervised pretraining strategy, yet its dependence on cohort composition remains underexplored. We present Contrasting by Augmented Patient Electrocardiograms (CAPE) foundation model and pretrain on four cohorts (n = 5,203,269), from diverse populations across three continents (North America, South America, Asia). We systematically assess how cohort demographics, health status, and population diversity influence the downstream performance for prediction tasks also including two additional cohorts from another continent (Europe). We find that downstream performance depends on the distributional properties of the pretraining cohort, including demographics and health status. Moreover, while pretraining with a multi-centre, demographically diverse cohort improves in-distribution accuracy, it reduces out-of-distribution (OOD) generalisation of our contrastive approach by encoding cohort-specific artifacts. To address this, we propose the In-Distribution Batch (IDB) strategy, which preserves intra-cohort consistency during pretraining, discourages learning of spurious cohort-specific features, and instead promotes clinically meaningful variability within cohorts. This leads to improved out-of-distribution robustness, with gains of 9–40% in downstream label prediction performance. This work provides insights into pretraining strategies for more clinically deployable and generalisable foundation models. Author summary: Artificial intelligence (AI) can learn from large collections of electrocardiograms (ECGs) and support the development of new diagnostic tools without relying on extensive manual annotation. However, it remains unclear how the data used to pretrain these models influences their ability to generalise across different patient populations.
Related
- Spatially guided translation from histology images to transcriptomic profiles using foundation model-driven contrastive learning
- Parasitism of an intertidal foundation species: Allometric scaling and environmental constraints on trematode distribution in barnacles across biological scales
- Predicting miRNA–Disease Associations Based on Contrastive Learning of Linear and Nonlinear Features
- CLIVeg: Contrastive Image-Vegetation Alignment for Botany-Aware Earth Observation Representations
- A global–local momentum contrastive framework with critical feature calibration for field plant disease identification in precision agriculture
BibTeX
@article{khattak2026data,
title = {{Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms}},
author = {Gul Rukh Khattak and Konstantinos Patlatzoglou and Joseph Barker and Libor Pastika and Boroumand Zeidaabadi and Aidan R. Birdi and Jiayu Huo and Ahmed El-Medany and Hesham Aggour and Yixiu Liang and Antonio H. Ribeiro and Jeffrey Annis and Antonio Luiz Pinho Ribeiro and Junbo Ge and Daniel B. Kramer and Jonathan W. Waks and Evan Brittain and Nicholas Peters and Fu Siong Ng and Arunashis Sau},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001623},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001623},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Data distribution impacts the performance and generalisability of contrastive learning-based foundation models of electrocardiograms
AU - Gul Rukh Khattak
AU - Konstantinos Patlatzoglou
AU - Joseph Barker
AU - Libor Pastika
AU - Boroumand Zeidaabadi
AU - Aidan R. Birdi
AU - Jiayu Huo
AU - Ahmed El-Medany
AU - Hesham Aggour
AU - Yixiu Liang
AU - Antonio H. Ribeiro
AU - Jeffrey Annis
AU - Antonio Luiz Pinho Ribeiro
AU - Junbo Ge
AU - Daniel B. Kramer
AU - Jonathan W. Waks
AU - Evan Brittain
AU - Nicholas Peters
AU - Fu Siong Ng
AU - Arunashis Sau
PY - 2026
DA - 2026/09/02
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001623
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001623
AB - Contrastive learning is a widely adopted self-supervised pretraining strategy, yet its dependence on cohort composition remains underexplored. We present Contrasting by Augmented Patient Electrocardiograms (CAPE) foundation model and pretrain on four cohorts (n = 5,203,269), from diverse populations across three continents (North America, South America, Asia). We systematically assess how cohort demographics, health status, and population diversity influence the downstream performance for prediction tasks also including two additional cohorts from another continent (Europe). We find that downstream performance depends on the distributional properties of the pretraining cohort, including demographics and health status. Moreover, while pretraining with a multi-centre, demographically diverse cohort improves in-distribution accuracy, it reduces out-of-distribution (OOD) generalisation of our contrastive approach by encoding cohort-specific artifacts. To address this, we propose the In-Distribution Batch (IDB) strategy, which preserves intra-cohort consistency during pretraining, discourages learning of spurious cohort-specific features, and instead promotes clinically meaningful variability within cohorts. This leads to improved out-of-distribution robustness, with gains of 9–40% in downstream label prediction performance. This work provides insights into pretraining strategies for more clinically deployable and generalisable foundation models. Author summary: Artificial intelligence (AI) can learn from large collections of electrocardiograms (ECGs) and support the development of new diagnostic tools without relying on extensive manual annotation. However, it remains unclear how the data used to pretrain these models influences their ability to generalise across different patient populations.
SN - 2767-3170
ER -