Benchmarking Classical Machine Learning and Deep Learning Models on the Cleveland Heart Disease Binary Classification Dataset
Abstract
Many high-stakes applications relied on binary classification, yet practitioners lacked equitable, methodical benchmarks that evaluated diverse model families under identical conditions. This study used the Cleveland Heart Disease tabular dataset to conduct a controlled, empirical benchmark of a regularised linear model (Ridge Classifier), tree-based ensembles (XGBoost, Random Forest), and a hybrid deep learning architecture combining Bidirectional Long Short-Term Memory and Convolutional Neural Network layers (BiLSTM-CNN) on a mildly imbalanced binary classification task. A strict protocol was applied, comprising uniform pre-processing, Synthetic Minority Oversampling Technique (SMOTE) applied to training folds only, a hybrid hold-out design with internal five-fold stratified crossvalidation on the training set (70%), and evaluation using accuracy, positive-class precision, recall, F1 score, and area under the curve (AUC) on a fixed test set (15%). Cross-validation results showed that no single model dominated every metric: the Ridge Classifier unexpectedly achieved the best AUC (0.8945), indicating strong ranking ability; Random Forest and XGBoost achieved the highest accuracy (81.67%); and the BiLSTM-CNN hybrid achieved the highest recall and F1 score (0.8478 and 0.8365, respectively). These findings indicated that, for this dataset, classical and ensemble approaches remained competitive with the more complex hybrid deep learning model, suggesting their continued relevance for similar small-scale, tabular binary classification tasks.
Related
- Reproducible benchmarking of four machine-learning classifiers on a large synthetic heart-disease competition dataset
- Statistical Machine Learning for Medicinal Plant Leaves Classification
- Comparing the use of supervised machine learning variable selection methods in the context of two-group classification in the psychological and health sciences
- Explainable computational framework for subtype classification and biomarker identification in head and neck squamous cell carcinoma using machine learning
- Reproducibility and Leakage-Controlled Evaluation of Machine-Learning Models for Chronic Kidney Disease Classification
BibTeX
@article{edmund2026benchmarking,
title = {{Benchmarking Classical Machine Learning and Deep Learning Models on the Cleveland Heart Disease Binary Classification Dataset}},
author = {Ajimah Nnabueze Edmund and Ikiomoye Douglas Emmanuel and Esenogho Ebenezer},
journal = {Journal of Information Systems and Informatics},
year = {2026},
month = oct,
doi = {10.63158/journalisi.v8i5.1794},
url = {https://doi.org/10.63158/journalisi.v8i5.1794},
note = {CC BY},
}
RIS
TY - JOUR
TI - Benchmarking Classical Machine Learning and Deep Learning Models on the Cleveland Heart Disease Binary Classification Dataset
AU - Ajimah Nnabueze Edmund
AU - Ikiomoye Douglas Emmanuel
AU - Esenogho Ebenezer
PY - 2026
DA - 2026/10/24
JO - Journal of Information Systems and Informatics
DO - 10.63158/journalisi.v8i5.1794
UR - https://doi.org/10.63158/journalisi.v8i5.1794
AB - Many high-stakes applications relied on binary classification, yet practitioners lacked equitable, methodical benchmarks that evaluated diverse model families under identical conditions. This study used the Cleveland Heart Disease tabular dataset to conduct a controlled, empirical benchmark of a regularised linear model (Ridge Classifier), tree-based ensembles (XGBoost, Random Forest), and a hybrid deep learning architecture combining Bidirectional Long Short-Term Memory and Convolutional Neural Network layers (BiLSTM-CNN) on a mildly imbalanced binary classification task. A strict protocol was applied, comprising uniform pre-processing, Synthetic Minority Oversampling Technique (SMOTE) applied to training folds only, a hybrid hold-out design with internal five-fold stratified crossvalidation on the training set (70%), and evaluation using accuracy, positive-class precision, recall, F1 score, and area under the curve (AUC) on a fixed test set (15%). Cross-validation results showed that no single model dominated every metric: the Ridge Classifier unexpectedly achieved the best AUC (0.8945), indicating strong ranking ability; Random Forest and XGBoost achieved the highest accuracy (81.67%); and the BiLSTM-CNN hybrid achieved the highest recall and F1 score (0.8478 and 0.8365, respectively). These findings indicated that, for this dataset, classical and ensemble approaches remained competitive with the more complex hybrid deep learning model, suggesting their continued relevance for similar small-scale, tabular binary classification tasks.
SN - 2656-5935
SN - 2656-4882
ER -