LEME: open large language models for ophthalmology with advanced reasoning and clinical validation
18 affiliations
- 1Department of Biomedical Informatics and Data Science, Yale School of Medicine, Yale University, New Haven, CT, USA
- 2Centre for Innovation and Precision Eye Health, Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- 3Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
- 4Department of Ophthalmology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
- 5Department of Ophthalmology and Visual Science, Yale School of Medicine, Yale University, New Haven, CT, USA
- 6Department of Computer Science, Korea University, Seoul, Republic of Korea
- 7Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore
- 8Department of Ophthalmology, National University Hospital, Singapore, Singapore
- 9Department of Computer Science, University of Liverpool, Liverpool, UK
- 10Byers Eye Institute, Stanford University, Stanford, CA, USA
- 11Institute of Health Informatics, University College London, London, UK
- 12Institute of Ophthalmology, University College London, London, UK
- 13Moorfields Eye Hospital, London, UK
- 14Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- 15Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA
- 16Research Informatics Office, Yale School of Medicine, Yale University, New Haven, CT, USA
- 17School of Biomedical Informatics, University of Texas Health Science Center, Houston, TX, USA
- 18Eye Academic Clinical Program (Eye ACP), Duke-NUS Medical School, Singapore, Singapore
Abstract
Large language models (LLMs) offer potential to alleviate the growing public health burden of eye disease; however, few models have been tailored for ophthalmology or evaluated against clinically relevant benchmarks and real-world workflows. Here, we present LEME, a suite of open-weight LLMs for ophthalmology developed via a two-stage framework: instruction tuning on 211,149 examples curated from authoritative sources, and reinforcement learning with 29,747 preference-labeled examples to enhance clinical reasoning and informativeness. LEME was evaluated under zero-shot settings using five curated benchmarks, covering question answering and patient-physician consultation. LEME outperformed all seven baselines (all p < 0.004), notably exceeding GPT-4o by 3.32% in average ROUGE-L. Furthermore, LEME was assessed on three downstream tasks using deidentified patient data. In answering patient queries, LEME received the highest overall ratings from attending clinicians; its completeness rating even surpassed that of the expert-written answers ( p = 0.015). For visual acuity extraction from clinical notes, LEME achieved the highest F1 scores, outperforming LLaMA-3 70B by 14.1% and Eye-LLaMA by 59.0%. Finally, in assessment and treatment plan generation, LEME achieved ratings approaching those of attending clinicians, demonstrating promise for ophthalmic decision support. All models, datasets, and code are made open to support further development and validation.
Related
- Clinical Reasoning in Veterinary Ophthalmology: A Pathophysiology-Centered Framework for Canine and Feline Clinical Practice.
- Influence of large language models on pediatricians’ clinical reasoning and decision-making process
- Calibrated Reliance on Large Language Models for Personalized Surgical Care: A Critical Narrative Review of Confidence, Clinical Reasoning and Decision Support
- Natural Language Processing in Nursing Practice for Clinical Reasoning and Workflow: Scoping Review
- SIGHT: A Methodological Appraisal Framework for Evaluating Large Language Models in Ophthalmology
BibTeX
@article{kim2026leme,
title = {{LEME: open large language models for ophthalmology with advanced reasoning and clinical validation}},
author = {Hyunjae Kim and Xuguang Ai and Sahana Srinivasan and Aidan Gilson and Maxwell Singer and Krithi Pushpanathan and Qianqian Xie and Jungwoo Park and Serina S. Applebaum and Gabriel Dawei Yang and Minjie Zou and David Ziyou Chen and Ke Zou and Soshian Sarrafpour and Ji Liu and Yu Yin and Jimin Huang and Quang Nguyen and Erping Long and Peixing Wan and Dianbo Liu and Richard Hintz and W. Jim Zheng and Sophia Ying Wang and Lucila Ohno-machado A and Hua Xu and Ron Afshari Adelman and Luciano V. Del Priore and Yih Chung Tham and Qingyu Chen},
journal = {npj Digital Medicine},
year = {2026},
month = sep,
doi = {10.1038/s41746-026-03104-4},
url = {https://doi.org/10.1038/s41746-026-03104-4},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - LEME: open large language models for ophthalmology with advanced reasoning and clinical validation
AU - Hyunjae Kim
AU - Xuguang Ai
AU - Sahana Srinivasan
AU - Aidan Gilson
AU - Maxwell Singer
AU - Krithi Pushpanathan
AU - Qianqian Xie
AU - Jungwoo Park
AU - Serina S. Applebaum
AU - Gabriel Dawei Yang
AU - Minjie Zou
AU - David Ziyou Chen
AU - Ke Zou
AU - Soshian Sarrafpour
AU - Ji Liu
AU - Yu Yin
AU - Jimin Huang
AU - Quang Nguyen
AU - Erping Long
AU - Peixing Wan
AU - Dianbo Liu
AU - Richard Hintz
AU - W. Jim Zheng
AU - Sophia Ying Wang
AU - Lucila Ohno-machado A
AU - Hua Xu
AU - Ron Afshari Adelman
AU - Luciano V. Del Priore
AU - Yih Chung Tham
AU - Qingyu Chen
PY - 2026
DA - 2026/09/25
JO - npj Digital Medicine
DO - 10.1038/s41746-026-03104-4
UR - https://doi.org/10.1038/s41746-026-03104-4
AB - Large language models (LLMs) offer potential to alleviate the growing public health burden of eye disease; however, few models have been tailored for ophthalmology or evaluated against clinically relevant benchmarks and real-world workflows. Here, we present LEME, a suite of open-weight LLMs for ophthalmology developed via a two-stage framework: instruction tuning on 211,149 examples curated from authoritative sources, and reinforcement learning with 29,747 preference-labeled examples to enhance clinical reasoning and informativeness. LEME was evaluated under zero-shot settings using five curated benchmarks, covering question answering and patient-physician consultation. LEME outperformed all seven baselines (all p < 0.004), notably exceeding GPT-4o by 3.32% in average ROUGE-L. Furthermore, LEME was assessed on three downstream tasks using deidentified patient data. In answering patient queries, LEME received the highest overall ratings from attending clinicians; its completeness rating even surpassed that of the expert-written answers ( p = 0.015). For visual acuity extraction from clinical notes, LEME achieved the highest F1 scores, outperforming LLaMA-3 70B by 14.1% and Eye-LLaMA by 59.0%. Finally, in assessment and treatment plan generation, LEME achieved ratings approaching those of attending clinicians, demonstrating promise for ophthalmic decision support. All models, datasets, and code are made open to support further development and validation.
SN - 2398-6352
ER -