Can artificial intelligence match refractive surgeons? : A comparative analysis of refractive surgery candidate selection
- 1Department of Ophthalmology, Başkent University, Istanbul, Türkiye
- 2Department of Ophthalmology, Fırat University, Elazığ, Türkiye.
Abstract
This study aimed to evaluate the accuracy of 2 large language models (LLMs) – ChatGPT-5.0 and Gemini Flash 2.5 – in assessing refractive-surgery eligibility and recommending appropriate procedures, using refractive-surgeon consensus as the reference standard. This retrospective, single-center study included the right eyes of 40 consecutive patients evaluated for refractive surgery at Başkent University Hospital (Istanbul, Türkiye). To avoid inter-eye correlation, only the right eye of each patient was included. Standardised refractive and corneal tomographic data obtained using the Pentacam HR were independently reviewed by 2 refractive surgeons and by the 2 LLMs. Each model received identical English-language prompts once per eye. Outputs were classified as “eligible,” “borderline,” or “ineligible”; for all binary statistical analyses, borderline assessments were grouped with the ineligible category. Recommended surgical procedures were recorded separately, with more than 2 procedure permitted for the same eye. Agreement between experts and between each LLM and expert consensus was assessed using Gwet’s agreement coefficient 1 (AC1) coefficient. Inter-expert agreement was good (AC1 = 0.78). Compared with expert consensus, ChatGPT-5.0 demonstrated very good agreement (AC1 = 0.92), outperforming Gemini Flash 2.5, which showed good agreement (AC1 = 0.78). Both LLMs accurately identified most “eligible” eyes; however, specificity and negative predictive values exhibited wide confidence intervals due to the limited number of “ineligible” cases. ChatGPT-5.0 showed higher sensitivity (92.1%) and negative predictive value (40.0%) than Gemini Flash 2.5 (81.6% and 22.2%, respectively). Notably, neither model recommended implantable collamer lens surgery, even when clinically appropriate. ChatGPT-5.0 demonstrated very good agreement with expert consensus and higher agreement than Gemini Flash 2.5 in refractive-surgery eligibility assessment. Although both LLMs show potential as adjunctive decision-support tools, the omission of large language model recommendations and prompt-dependent variability limit their suitability for autonomous clinical use. Further optimization using comprehensive refractive-surgery datasets is required to improve clinical reliability.
Related
- Evaluation of Tooth Shade Selection With Conventional Methods and Artificial Intelligence-Integrated Methods: A Comparative Study
- Artificial Intelligence Applications in Laser-Assisted Refractive Surgery—A Comprehensive Review
- OP–249 A comparative evaluation of artificial intelligence models for adolescent-centered scoliosis questions
- Artificial intelligence-enabled healthcare access and user experience: comparative institutional analysis and survey evidence from China
- Quality, transparency, and readability of artificial intelligence chatbot responses to fibrous dysplasia-related searches: a cross-sectional comparative study
BibTeX
@article{iritas2026artificial,
title = {{Can artificial intelligence match refractive surgeons? : A comparative analysis of refractive surgery candidate selection}},
author = {İlter İritaş and Murat Erdağ},
journal = {Medicine},
year = {2026},
month = oct,
doi = {10.1097/md.0000000000051367},
url = {https://europepmc.org/article/PMC/PMC13727163},
note = {CC BY-NC},
}
RIS
TY - JOUR
TI - Can artificial intelligence match refractive surgeons? : A comparative analysis of refractive surgery candidate selection
AU - İlter İritaş
AU - Murat Erdağ
PY - 2026
DA - 2026/10/30
JO - Medicine
DO - 10.1097/md.0000000000051367
UR - https://europepmc.org/article/PMC/PMC13727163
AB - This study aimed to evaluate the accuracy of 2 large language models (LLMs) – ChatGPT-5.0 and Gemini Flash 2.5 – in assessing refractive-surgery eligibility and recommending appropriate procedures, using refractive-surgeon consensus as the reference standard. This retrospective, single-center study included the right eyes of 40 consecutive patients evaluated for refractive surgery at Başkent University Hospital (Istanbul, Türkiye). To avoid inter-eye correlation, only the right eye of each patient was included. Standardised refractive and corneal tomographic data obtained using the Pentacam HR were independently reviewed by 2 refractive surgeons and by the 2 LLMs. Each model received identical English-language prompts once per eye. Outputs were classified as “eligible,” “borderline,” or “ineligible”; for all binary statistical analyses, borderline assessments were grouped with the ineligible category. Recommended surgical procedures were recorded separately, with more than 2 procedure permitted for the same eye. Agreement between experts and between each LLM and expert consensus was assessed using Gwet’s agreement coefficient 1 (AC1) coefficient. Inter-expert agreement was good (AC1 = 0.78). Compared with expert consensus, ChatGPT-5.0 demonstrated very good agreement (AC1 = 0.92), outperforming Gemini Flash 2.5, which showed good agreement (AC1 = 0.78). Both LLMs accurately identified most “eligible” eyes; however, specificity and negative predictive values exhibited wide confidence intervals due to the limited number of “ineligible” cases. ChatGPT-5.0 showed higher sensitivity (92.1%) and negative predictive value (40.0%) than Gemini Flash 2.5 (81.6% and 22.2%, respectively). Notably, neither model recommended implantable collamer lens surgery, even when clinically appropriate. ChatGPT-5.0 demonstrated very good agreement with expert consensus and higher agreement than Gemini Flash 2.5 in refractive-surgery eligibility assessment. Although both LLMs show potential as adjunctive decision-support tools, the omission of large language model recommendations and prompt-dependent variability limit their suitability for autonomous clinical use. Further optimization using comprehensive refractive-surgery datasets is required to improve clinical reliability.
SN - 0304-5412
SN - 1536-5964
SN - 0025-7974
ER -