Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field
4 affiliations
- 1School of Laboratory Medicine, Jilin Medical University
- 2School of Artificial Intelligence, Jiangnan University
- 3Department of Cardiology, Jilin Central Hospital
- 4School of Clinical Medicine, Jilin Medical University
Abstract
Background: Cardiovascular diseases (CVDs) remain a major global health burden, and electrocardiogram (ECG) classification is an important component of automated cardiac screening. However, conventional deep learning approaches may lose complementary information when operating on a single representation of the signal. This study aims to develop a multimodal method for normal-versus-abnormal ECG heartbeat classification under patient-independent evaluation.
Methods: We propose a multi-modal neural network that integrates raw ECG signals and their Gramian Angular Field (GAF) image representations. This dual-modal strategy enables the model to capture complementary time-series and two-dimensional structural features. The classifier combines lightweight multi-head attention with a residual feed-forward network, and the ECG, GASF, and fused representations provide coordinated prediction pathways followed by majority voting. To address generalization and patient-level leakage concerns, all formal experiments used strict patient-level five-fold cross-validation with patient-disjoint training, validation, and test partitions. Checkpoints were selected using validation ROC AUC, and the same fixed patient partitions were used for the ablation study and the BiLSTM/CNN + Transformer comparisons.
Results: Under the strict patient-level five-fold protocol, the Full Multimodal model achieved a ROC AUC of 0.996 and an accuracy (ACC) of 0.983. In the protocol-matched comparison, ECG-only achieved 0.815/0.846, BiLSTM 0.758/0.842, and CNN + Transformer 0.854/0.884 for ROC AUC/ACC, respectively. The ablation variants all showed lower ROC AUC and ACC than the complete model, supporting the contribution of the coordinated multimodal framework.
Conclusion: The proposed framework provides a patient-independent approach to normal-versus-abnormal ECG heartbeat classification by coordinating raw ECG and GASF representations. The revised patient-level cross-validation, component-level ablation, protocol-matched comparison, and updated literature context provide a more rigorous evaluation of the framework while supporting further validation in broader and external ECG cohorts.
Related
- Comparative efficacy of digital health intervention delivery modalities on quality of life, anxiety, and depression in coronary heart disease: a systematic review and network meta-analysis
- Psychological Network Structure of Kinesiophobia and Pain Catastrophizing in Patients with Stable Coronary Heart Disease: An Exploratory Cross-Sectional Dimension-Level Network Analysis.
- Mapping the Dynamics of Depressive and Anxiety symptoms in Coronary Artery Disease over two years: A Network Analysis
- Antiplatelet monotherapy maintenance strategies after percutaneous coronary intervention in patients with coronary artery disease at high bleeding risk: a network meta-analysis
- A multi-class fundus disease classification framework based on self-supervised contrastive learning and dual-branch network
BibTeX
@article{jiang2026multimodal,
title = {{Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field}},
author = {Yong Jiang and Xiang Pan and Ying Zhao and Xin-hua Zou and Han-qi Yang and Kai-qi Li and Yan-ting Zhu and Jian-zhe Huang and Shi-yun Nie and Jia-yin Wang and Ting-miao Tang},
journal = {Frontiers in Cardiovascular Medicine},
year = {2026},
month = nov,
doi = {10.3389/fcvm.2026.1844501},
url = {https://www.frontiersin.org/articles/10.3389/fcvm.2026.1844501/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Multi-modal neural network for ECG signal classification in coronary artery disease using gramian angular field
AU - Yong Jiang
AU - Xiang Pan
AU - Ying Zhao
AU - Xin-hua Zou
AU - Han-qi Yang
AU - Kai-qi Li
AU - Yan-ting Zhu
AU - Jian-zhe Huang
AU - Shi-yun Nie
AU - Jia-yin Wang
AU - Ting-miao Tang
PY - 2026
DA - 2026/11/16
JO - Frontiers in Cardiovascular Medicine
DO - 10.3389/fcvm.2026.1844501
UR - https://www.frontiersin.org/articles/10.3389/fcvm.2026.1844501/full
AB - Background: Cardiovascular diseases (CVDs) remain a major global health burden, and electrocardiogram (ECG) classification is an important component of automated cardiac screening. However, conventional deep learning approaches may lose complementary information when operating on a single representation of the signal. This study aims to develop a multimodal method for normal-versus-abnormal ECG heartbeat classification under patient-independent evaluation. Methods: We propose a multi-modal neural network that integrates raw ECG signals and their Gramian Angular Field (GAF) image representations. This dual-modal strategy enables the model to capture complementary time-series and two-dimensional structural features. The classifier combines lightweight multi-head attention with a residual feed-forward network, and the ECG, GASF, and fused representations provide coordinated prediction pathways followed by majority voting. To address generalization and patient-level leakage concerns, all formal experiments used strict patient-level five-fold cross-validation with patient-disjoint training, validation, and test partitions. Checkpoints were selected using validation ROC AUC, and the same fixed patient partitions were used for the ablation study and the BiLSTM/CNN + Transformer comparisons. Results: Under the strict patient-level five-fold protocol, the Full Multimodal model achieved a ROC AUC of 0.996 and an accuracy (ACC) of 0.983. In the protocol-matched comparison, ECG-only achieved 0.815/0.846, BiLSTM 0.758/0.842, and CNN + Transformer 0.854/0.884 for ROC AUC/ACC, respectively. The ablation variants all showed lower ROC AUC and ACC than the complete model, supporting the contribution of the coordinated multimodal framework. Conclusion: The proposed framework provides a patient-independent approach to normal-versus-abnormal ECG heartbeat classification by coordinating raw ECG and GASF representations. The revised patient-level cross-validation, component-level ablation, protocol-matched comparison, and updated literature context provide a more rigorous evaluation of the framework while supporting further validation in broader and external ECG cohorts.
SN - 2297-055X
ER -