Diabetic retinopathy feature utilisation in 54 convolutional neural networks: a heatmap analysis
- 1University of Canberra
Abstract
Purpose: To investigate how artificial intelligence (AI) systems detect referable diabetic retinopathy (DR) from retinal photographs by analysing heatmap patterns and determining their overlap with DR features. Methods: Fifty-four AI systems were developed using 27 backbone architectures, with each implemented as both binary-referable and multi-class grading models based on the International Clinical Diabetic Retinopathy (ICDR) grading scale. Models were trained on DDR, BRSET and Kaggle datasets. After training, each model analysed 749 images with DR feature annotations, with Grad-CAM, Integrated Gradients, LIME and SHAP heatmaps generated and compared to pixel-level annotations of microaneurysms, haemorrhages, exudates, cotton wool spots, venous beading, intraretinal microvascular abnormalities and neovascularisation. Results: All models achieved acceptable predictive performance (AUROC >0.8 for most architectures). Heatmap analysis revealed consistent attention to the central and temporal macular region. Exudates and cotton wool spots were highlighted most frequently by the heatmaps, with venous beading and neovascularisation at the disc showing poor overall coverage for binary referable classifiers. Models grading per the ICDR scale demonstrated higher coverage for all features. Substantial variability was observed between architectures, suggesting different feature detection capabilities. Interestingly, the heatmap analysis indicated that the models may be using different features to the ICDR grading scale definitions. Conclusion: AI models do not uniformly correlate with all DR features when detecting referable DR, limiting their predictive performance in unusual presentations. Heatmap aggregation analysis provides a scalable method for analysing model behaviour, allowing strengths and weaknesses to be assessed. These findings may help improve clinician's trust and acceptance of AI.
Related
- CBAM-enhanced convolutional networks for diabetic retinopathy classification in the ODIR dataset
- EIF1 coordinates transcriptomic and splicing networks associated with cell cycle dysregulation in diabetic retinopathy
- Feature-aware attention convolutional neural network for efficient pulmonary fibrosis classification from chest CT images with FPGA deployment
- Successful management of high-risk proliferative diabetic retinopathy during pregnancy with intravitreal faricimab
- ISG15+ Müller glial subtype contributes to early blood–retinal barrier breakdown in diabetic retinopathy
BibTeX
@misc{murphy2026diabetic,
title = {{Diabetic retinopathy feature utilisation in 54 convolutional neural networks: a heatmap analysis}},
author = {T. I. Murphy and J. A. Armitage},
howpublished = {medRxiv},
year = {2026},
month = jun,
doi = {10.64898/2026.06.19.26356040},
url = {https://www.medrxiv.org/content/10.64898/2026.06.19.26356040v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - Diabetic retinopathy feature utilisation in 54 convolutional neural networks: a heatmap analysis
AU - T. I. Murphy
AU - J. A. Armitage
PY - 2026
DA - 2026/06/19
PB - medRxiv
DO - 10.64898/2026.06.19.26356040
UR - https://www.medrxiv.org/content/10.64898/2026.06.19.26356040v2
AB - Purpose: To investigate how artificial intelligence (AI) systems detect referable diabetic retinopathy (DR) from retinal photographs by analysing heatmap patterns and determining their overlap with DR features. Methods: Fifty-four AI systems were developed using 27 backbone architectures, with each implemented as both binary-referable and multi-class grading models based on the International Clinical Diabetic Retinopathy (ICDR) grading scale. Models were trained on DDR, BRSET and Kaggle datasets. After training, each model analysed 749 images with DR feature annotations, with Grad-CAM, Integrated Gradients, LIME and SHAP heatmaps generated and compared to pixel-level annotations of microaneurysms, haemorrhages, exudates, cotton wool spots, venous beading, intraretinal microvascular abnormalities and neovascularisation. Results: All models achieved acceptable predictive performance (AUROC >0.8 for most architectures). Heatmap analysis revealed consistent attention to the central and temporal macular region. Exudates and cotton wool spots were highlighted most frequently by the heatmaps, with venous beading and neovascularisation at the disc showing poor overall coverage for binary referable classifiers. Models grading per the ICDR scale demonstrated higher coverage for all features. Substantial variability was observed between architectures, suggesting different feature detection capabilities. Interestingly, the heatmap analysis indicated that the models may be using different features to the ICDR grading scale definitions. Conclusion: AI models do not uniformly correlate with all DR features when detecting referable DR, limiting their predictive performance in unusual presentations. Heatmap aggregation analysis provides a scalable method for analysing model behaviour, allowing strengths and weaknesses to be assessed. These findings may help improve clinician's trust and acceptance of AI.
ER -