Performance Assessment of an Artificial Intelligence System Used for Case Detection in a Large Community-based Diabetic Retinopathy Screening Program: A Retrospective Diagnostic Accuracy Validation Study.
- 1Ilocos Training and Regional Medical Center, La Union, Philippines.
- 2Department of Ophthalmology and Visual Sciences, Sentro Oftalmologico Jose Rizal, Philippine General Hospital, University of the Philippines Manila, Manila, Philippines.
Abstract
In the Philippines, Artificial Intelligence (AI) systems used for diabetic retinopathy (DR) screening are still an emerging technology. There are several AI and Deep Learning Systems (DLS) developed and licensed abroad that are currently undergoing FDA approval in the Philippines. The challenge is to validate these systems on Filipino diabetic cohorts within the uncontrolled environment of a community-based setting before it could be adapted and integrated into routine DR screening programs locally. This study aimed to determine the performance of an AI system as a population-based mass screening tool to detect referable cases of DR. This study was designed as a retrospective diagnostic accuracy validation study using fundus images obtained from a community-based screening program by the Ilocos Training & Regional Medical Center Eye Mobile Team in the province of La Union. The diagnostic accuracy of the iCare Illume System (Thirona RetCAD + iCare DRS plus) in detecting any stage DR (ADR) and referable DR (RDR) was validated against a reference standard of two-field fundus photographs read by expert human graders (retina specialists). The diagnostic performance was evaluated using various metrics, including Accuracy, Sensitivity (SN), Specificity (SP), Negative Predictive Value (NPV), Positive Predictive Value (PPV), Negative Likelihood Ratio (LR-), and Positive Likelihood Ratios (LR+). A total of 284 eyes (142 diabetic patients, 717 images) were included in the study. Human expert graders classified 187 eyes (65.85%) as no apparent DR while 77 eyes (27.11%) as ADR. 63 eyes (22.18%) were considered RDR. The iCare Illume AI system achieved an accuracy of 83.21 % and 92.37% for ADR and RDR, respectively. The SN, SP, NPV, PPV, LR-, and LR+ for ADR were 92.11%, 79.57%, 96.1%, 64.81%, 0.10, and 4.50, respectively. The SN, SP, NPV, PPV, LR-, and LR+ for RDR were 79.03%, 96.50%, 93.69%, 87.50%, 0.22, and 22.58, respectively. Technical failure rate was 0.70% (99.3% Imageability). The iCare Illume system exhibits strong diagnostic accuracy in screening for referable DR in our cohort of Filipino Diabetics. Although its sensitivity is marginally below the UK NICE guideline, the system demonstrates high specificity, negative predictive value, and positive predictive value. This combination highlights its potential as a valuable tool for diabetic retinopathy screening in the Philippines. Future research should focus on refining local AI validation protocols, optimizing data sets of commercially available AI systems, assessing the impact on patient outcomes and disease burden, and evaluating cost-effectiveness.
Related
- CT-Referenced Diagnostic Performance and Complementary Error Patterns of Artificial Intelligence Versus a Single Emergency Physician in Chest Radiograph Interpretation: A Retrospective Paired Diagnostic Accuracy Study
- Artificial intelligence-supported diabetic retinopathy screening in Tanzania: a randomised controlled trial
- Multimodal artificial intelligence assessment of simulated cardiac arrest performance: a pilot study
- Generative Artificial Intelligence Use and Applied Higher-Order Cognitive Performance Among Health Informatics Students: A Multicenter Cross-Sectional Case-Based Assessment Study.
- The effect of artificial intelligence-driven, computed tomography-based assessment of body composition parameters on 28-day mortality of patients with severe fever with thrombocytopenia syndrome: a retrospective cohort study.
BibTeX
@article{eduardo2026performance,
title = {{Performance Assessment of an Artificial Intelligence System Used for Case Detection in a Large Community-based Diabetic Retinopathy Screening Program: A Retrospective Diagnostic Accuracy Validation Study.}},
author = {Maria Kristina Dg Eduardo and Junn R Pajarillo},
journal = {Acta medica Philippina},
year = {2026},
month = sep,
doi = {10.47895/amp.v60i17.13381},
url = {https://europepmc.org/article/MED/42952879},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Performance Assessment of an Artificial Intelligence System Used for Case Detection in a Large Community-based Diabetic Retinopathy Screening Program: A Retrospective Diagnostic Accuracy Validation Study.
AU - Maria Kristina Dg Eduardo
AU - Junn R Pajarillo
PY - 2026
DA - 2026/09/15
JO - Acta medica Philippina
DO - 10.47895/amp.v60i17.13381
UR - https://europepmc.org/article/MED/42952879
AB - In the Philippines, Artificial Intelligence (AI) systems used for diabetic retinopathy (DR) screening are still an emerging technology. There are several AI and Deep Learning Systems (DLS) developed and licensed abroad that are currently undergoing FDA approval in the Philippines. The challenge is to validate these systems on Filipino diabetic cohorts within the uncontrolled environment of a community-based setting before it could be adapted and integrated into routine DR screening programs locally. This study aimed to determine the performance of an AI system as a population-based mass screening tool to detect referable cases of DR. This study was designed as a retrospective diagnostic accuracy validation study using fundus images obtained from a community-based screening program by the Ilocos Training & Regional Medical Center Eye Mobile Team in the province of La Union. The diagnostic accuracy of the iCare Illume System (Thirona RetCAD + iCare DRS plus) in detecting any stage DR (ADR) and referable DR (RDR) was validated against a reference standard of two-field fundus photographs read by expert human graders (retina specialists). The diagnostic performance was evaluated using various metrics, including Accuracy, Sensitivity (SN), Specificity (SP), Negative Predictive Value (NPV), Positive Predictive Value (PPV), Negative Likelihood Ratio (LR-), and Positive Likelihood Ratios (LR+). A total of 284 eyes (142 diabetic patients, 717 images) were included in the study. Human expert graders classified 187 eyes (65.85%) as no apparent DR while 77 eyes (27.11%) as ADR. 63 eyes (22.18%) were considered RDR. The iCare Illume AI system achieved an accuracy of 83.21 % and 92.37% for ADR and RDR, respectively. The SN, SP, NPV, PPV, LR-, and LR+ for ADR were 92.11%, 79.57%, 96.1%, 64.81%, 0.10, and 4.50, respectively. The SN, SP, NPV, PPV, LR-, and LR+ for RDR were 79.03%, 96.50%, 93.69%, 87.50%, 0.22, and 22.58, respectively. Technical failure rate was 0.70% (99.3% Imageability). The iCare Illume system exhibits strong diagnostic accuracy in screening for referable DR in our cohort of Filipino Diabetics. Although its sensitivity is marginally below the UK NICE guideline, the system demonstrates high specificity, negative predictive value, and positive predictive value. This combination highlights its potential as a valuable tool for diabetic retinopathy screening in the Philippines. Future research should focus on refining local AI validation protocols, optimizing data sets of commercially available AI systems, assessing the impact on patient outcomes and disease burden, and evaluating cost-effectiveness.
SN - 0001-6071
SN - 2094-9278
ER -