Evaluating an AI-assisted triage workflow for retinal diseases
- 1VUNO Inc., Seoul, Korea
- 2Department of Ophthalmology, Seoul National University Hospital, Seoul, Korea
- 3Mass Eye and Ear and the Department of Ophthalmology at Harvard Medical School, Boston, Massachusetts, United States of America
Abstract
The clinical value of artificial intelligence (AI) in fundus photography depends on workflow efficiency as well as diagnostic performance. We evaluated an AI-assisted negative-screening workflow for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) using 6,904 color fundus photographs independently graded by three retina specialists. In this workflow, AI-negative images were classified as negative without ophthalmologist review, whereas AI-positive images were referred for human interpretation. For each reader–disease pair, the reference standard was agreement between the other two readers; discordant cases were excluded (DR, 1.2%–2.3%; RVO, 0.3%–0.6%; AMD, 6.1%–11.6%). The workflow reduced direct ophthalmologist review to 7.3%–8.0% of images for DR, 11.7%–11.9% for RVO, and 13.5%–16.8% for AMD. For DR, sensitivity decreased significantly (0.9168 to 0.8838; difference, -0.0330; 95% confidence interval [CI], -0.0495 to -0.0165; p < 0.001), whereas specificity did not change significantly (0.9961 to 0.9983; p = 0.183). For RVO, sensitivity was unchanged (0.9807) and specificity did not change significantly (0.9985 to 0.9988; p = 0.320). For AMD, sensitivity decreased significantly (0.8570 to 0.8445; difference, -0.0124; 95% CI, -0.0215 to -0.0033; p = 0.008), whereas specificity did not change significantly (0.9685 to 0.9843; p = 0.324). In an exploratory image-level analysis for at least one target disease, review decreased to 27.7%–30.1%; sensitivity changed from 0.9122 to 0.9039 (difference, -0.0083; 95% CI, -0.0131 to -0.0035; p < 0.001) and specificity from 0.9659 to 0.9804 (p = 0.334). AI-assisted negative-screening can reduce ophthalmologist workload across multiple retinal diseases, but with disease-specific safety trade-offs: because AI-negative images are not reviewed, reduced sensitivity for DR and AMD means a small proportion of true-positive cases would go undetected, potentially delaying diagnosis and treatment for sight-threatening disease. These results support disease-specific implementation strategies that balance workload reduction against missed positive cases. Author summary: Screening retinal photographs for common eye diseases is important, but reviewing every image requires specialist time. Many studies of artificial intelligence (AI) ask whether software can diagnose diseases on its own. We asked a different question: can it make screening more efficient when used before an ophthalmologist reviews an image? We tested a workflow in which AI software first examined 6,904 retinal photographs for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD). Images classified as negative were not reviewed further, while positive cases were referred to an ophthalmologist. This approach reduced direct specialist review to about 7%–8% of images for diabetic retinopathy, about 12% for retinal vein occlusion, and about 14%–17% for age-related macular degeneration. For RVO, performance was essentially unchanged compared with human-only reading. For DR and AMD, the workflow missed slightly more positive cases, with improved or unchanged specificity. Overall, these findings suggest that AI may be most useful as a triage tool in retinal screening, helping specialists focus on higher-risk images while balancing disease-specific trade-offs.
Related
- Evaluating an LLM-assisted workflow for clinical documentation: A pilot randomized controlled trial on time and quality
- Evaluating a Large Language Model for Emergency Triage Using Synthetic Clinical Dialogues: Toward a Multimodal Clinical Safety Framework
- AI-Assisted Triage for Diabetic Retinopathy Integrated into Routine Diabetes Care: A Real-World Diagnostic Accuracy Study of the DAIRET® Algorithm
- Evaluating the real-world accessibility of open-source deep learning models for retinal vessel segmentation in optical coherence tomography angiography
- Choriocapillaris Dysfunction in Retinal Diseases: Multimodal Imaging Biomarkers, Pathophysiological Insights, and Emerging Clinical Applications
BibTeX
@article{park2026evaluating,
title = {{Evaluating an AI-assisted triage workflow for retinal diseases}},
author = {Doohyun Park and Richul Oh and Jihyeon Baek and Kunho Bae},
journal = {PLOS Digital Health},
year = {2026},
month = sep,
doi = {10.1371/journal.pdig.0001700},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001700},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Evaluating an AI-assisted triage workflow for retinal diseases
AU - Doohyun Park
AU - Richul Oh
AU - Jihyeon Baek
AU - Kunho Bae
PY - 2026
DA - 2026/09/15
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001700
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001700
AB - The clinical value of artificial intelligence (AI) in fundus photography depends on workflow efficiency as well as diagnostic performance. We evaluated an AI-assisted negative-screening workflow for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) using 6,904 color fundus photographs independently graded by three retina specialists. In this workflow, AI-negative images were classified as negative without ophthalmologist review, whereas AI-positive images were referred for human interpretation. For each reader–disease pair, the reference standard was agreement between the other two readers; discordant cases were excluded (DR, 1.2%–2.3%; RVO, 0.3%–0.6%; AMD, 6.1%–11.6%). The workflow reduced direct ophthalmologist review to 7.3%–8.0% of images for DR, 11.7%–11.9% for RVO, and 13.5%–16.8% for AMD. For DR, sensitivity decreased significantly (0.9168 to 0.8838; difference, -0.0330; 95% confidence interval [CI], -0.0495 to -0.0165; p < 0.001), whereas specificity did not change significantly (0.9961 to 0.9983; p = 0.183). For RVO, sensitivity was unchanged (0.9807) and specificity did not change significantly (0.9985 to 0.9988; p = 0.320). For AMD, sensitivity decreased significantly (0.8570 to 0.8445; difference, -0.0124; 95% CI, -0.0215 to -0.0033; p = 0.008), whereas specificity did not change significantly (0.9685 to 0.9843; p = 0.324). In an exploratory image-level analysis for at least one target disease, review decreased to 27.7%–30.1%; sensitivity changed from 0.9122 to 0.9039 (difference, -0.0083; 95% CI, -0.0131 to -0.0035; p < 0.001) and specificity from 0.9659 to 0.9804 (p = 0.334). AI-assisted negative-screening can reduce ophthalmologist workload across multiple retinal diseases, but with disease-specific safety trade-offs: because AI-negative images are not reviewed, reduced sensitivity for DR and AMD means a small proportion of true-positive cases would go undetected, potentially delaying diagnosis and treatment for sight-threatening disease. These results support disease-specific implementation strategies that balance workload reduction against missed positive cases. Author summary: Screening retinal photographs for common eye diseases is important, but reviewing every image requires specialist time. Many studies of artificial intelligence (AI) ask whether software can diagnose diseases on its own. We asked a different question: can it make screening more efficient when used before an ophthalmologist reviews an image? We tested a workflow in which AI software first examined 6,904 retinal photographs for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD). Images classified as negative were not reviewed further, while positive cases were referred to an ophthalmologist. This approach reduced direct specialist review to about 7%–8% of images for diabetic retinopathy, about 12% for retinal vein occlusion, and about 14%–17% for age-related macular degeneration. For RVO, performance was essentially unchanged compared with human-only reading. For DR and AMD, the workflow missed slightly more positive cases, with improved or unchanged specificity. Overall, these findings suggest that AI may be most useful as a triage tool in retinal screening, helping specialists focus on higher-risk images while balancing disease-specific trade-offs.
SN - 2767-3170
ER -