A multi task deep learning framework for automated health condition prediction from retinal fundus images using hybrid attention and explainable AI
Abstract
Although retinal fundus imaging is a non-invasive tool to assess systemic and fundus pathology, current deep learning algorithms are mostly single-task, lack clinical interpretability and are unable to handle acquisition variation between different domains. These constraints limit their clinical utility: when a patient comes to the clinic for fundus screening, they could also be diabetic, have glaucoma, AMD, and increased cardiovascular risk, with each disease having retinal vascular biomarkers that require a patient to be tested for in the clinic, which must be managed within a framework of common, interpretable, and generalizable models rather than a pipeline of independent models. To solve these problems, we suggest a multi-task deep learning network that simultaneously classifies DR severity (five-class), glaucoma likelihood (binary), AMD stage (four-class) and cardiovascular risk (binary) from a single retinal fundus image using a single forward pass. It adopts an EfficientNet-B4 convolutional backbone pretrained on the ImageNet dataset, a six-layer multi-head self-attention Transformer encoder (d_model = 512, h = 8), a novel dual-branch spatial-channel attention module that builds on, but significantly expands, the CBAM module, and four task-specific output heads that are optimized using the homoscedastic uncertainty-weighted multi-task loss following the formulation of Kendall et al. Per-task anatomical explanations are produced by back propagating the CNN backbone via Grad-CAM in the anatomical space. The proposed model was evaluated on EyePACS, APTOS 2019, Messidor-2 and RIM-ONE v3 and achieved an accuracy of 94.7%, AUC = 0.978, sensitivity = 93.2% and specificity = 95.8% while being statistically better than all single task models (DeLong test, p < 0.01, Bonferroni corrected). Under the reported experimental setting, the zero-shot cross-dataset evaluation achieves AUC ≥ 0.951, which is promising for its performance across datasets. Grad-CAM saliency maps are anatomically similar to established clinical grading criteria for each disease. The four task-specific heads share a single architecture and produce four outputs in one forward pass, but no individual patient in the training corpora had verified ground-truth labels for all four conditions simultaneously; the model is trained on partially labelled, multi-source data via a masked multi-task loss, and the cardiovascular output is an AVR-derived image-based surrogate rather than a validated clinical cardiovascular diagnosis. This proposed framework could be considered for an additional potential step toward the future when it comes to multi-disease fundus screening with an interpretable single-model solution, if it is assessed and approved at an independent multi-centre level and by regulatory bodies.
Related
- Hybrid Explainable Deep Ensemble Framework with self-attention mechanism for early prediction of depression severity using PHQ-9 clinical and behavioral data
- CERVEX: A Foundation-Model Framework With Built-In Explainable AI for Automated Cervical Cytology Classification From Pap Smear Images
- Vehicle lane change prediction with explainable soft mask attention mechanism and heterogeneous information encoder
- An Integrated explainable machine learning framework for pavement condition assessment and maintenance prioritization: A Saudi Arabian case study
- Explainable AI framework for Alzheimer’s disease prediction using multimodal data
BibTeX
@article{tenali2026multi,
title = {{A multi task deep learning framework for automated health condition prediction from retinal fundus images using hybrid attention and explainable AI}},
author = {Ravi Kumar Tenali and Anitha Potnuri},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-74871-z},
url = {https://www.nature.com/articles/s41598-026-74871-z},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - A multi task deep learning framework for automated health condition prediction from retinal fundus images using hybrid attention and explainable AI
AU - Ravi Kumar Tenali
AU - Anitha Potnuri
PY - 2026
DA - 2026/10/17
JO - Scientific Reports
DO - 10.1038/s41598-026-74871-z
UR - https://www.nature.com/articles/s41598-026-74871-z
AB - Although retinal fundus imaging is a non-invasive tool to assess systemic and fundus pathology, current deep learning algorithms are mostly single-task, lack clinical interpretability and are unable to handle acquisition variation between different domains. These constraints limit their clinical utility: when a patient comes to the clinic for fundus screening, they could also be diabetic, have glaucoma, AMD, and increased cardiovascular risk, with each disease having retinal vascular biomarkers that require a patient to be tested for in the clinic, which must be managed within a framework of common, interpretable, and generalizable models rather than a pipeline of independent models. To solve these problems, we suggest a multi-task deep learning network that simultaneously classifies DR severity (five-class), glaucoma likelihood (binary), AMD stage (four-class) and cardiovascular risk (binary) from a single retinal fundus image using a single forward pass. It adopts an EfficientNet-B4 convolutional backbone pretrained on the ImageNet dataset, a six-layer multi-head self-attention Transformer encoder (d_model = 512, h = 8), a novel dual-branch spatial-channel attention module that builds on, but significantly expands, the CBAM module, and four task-specific output heads that are optimized using the homoscedastic uncertainty-weighted multi-task loss following the formulation of Kendall et al. Per-task anatomical explanations are produced by back propagating the CNN backbone via Grad-CAM in the anatomical space. The proposed model was evaluated on EyePACS, APTOS 2019, Messidor-2 and RIM-ONE v3 and achieved an accuracy of 94.7%, AUC = 0.978, sensitivity = 93.2% and specificity = 95.8% while being statistically better than all single task models (DeLong test, p < 0.01, Bonferroni corrected). Under the reported experimental setting, the zero-shot cross-dataset evaluation achieves AUC ≥ 0.951, which is promising for its performance across datasets. Grad-CAM saliency maps are anatomically similar to established clinical grading criteria for each disease. The four task-specific heads share a single architecture and produce four outputs in one forward pass, but no individual patient in the training corpora had verified ground-truth labels for all four conditions simultaneously; the model is trained on partially labelled, multi-source data via a masked multi-task loss, and the cardiovascular output is an AVR-derived image-based surrogate rather than a validated clinical cardiovascular diagnosis. This proposed framework could be considered for an additional potential step toward the future when it comes to multi-disease fundus screening with an interpretable single-model solution, if it is assessed and approved at an independent multi-centre level and by regulatory bodies.
SN - 2045-2322
ER -