Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties
9 affiliations
- 1Department of Ophthalmology, Midwestern University College of Osteopathic Medicine, Phoenix, USA
- 2Department of Ophthalmology, Creighton University College of Medicine, Phoenix, USA
- 3Department of Ophthalmology, Michigan State University College of Osteopathic Medicine, Warren, USA
- 4Department of Ophthalmology, Albany Medical College, New York, USA
- 5Department of Ophthalmology, Rocky Vista University College of Osteopathic Medicine, St. George, USA
- 6Department of Ophthalmology, Hoopes Moshirfar Research Center, Draper, USA
- 7Department of Ophthalmology, Hoopes Vision, Draper, USA
- 8John A. Moran Eye Center, University of Utah School of Medicine, Salt Lake City, USA
- 9Eye Banking and Corneal Transplantation, Utah Lions Eye Bank, Murray, USA
Abstract
Purpose
As the adoption of large language models (LLMs) accelerates in healthcare, the evaluation of their accuracy must accelerate in tandem. This study compares six LLMs' ability to generate accurate, thorough, and high-quality scientific technical writing in seven areas compared to a human-written control.
Methods
Four prompts were created in ophthalmic subspecialties and presented to ChatGPT (4.0), Gemini (2.5 Pro), DeepSeek (DeepSeek-Prover-V2-671B), Grok (3.0), Microsoft Copilot, and OpenEvidence (2.0). Each model's performance was analyzed based on seven factors: medical accuracy, thoroughness and comprehensiveness, organization, literature support, speech efficiency, citation classification, and readability. Data analysis was conducted in Excel, graphed, and compared to a control.
Results
The control texts outperformed each of the artificial intelligence (AI) models in medical accuracy, thoroughness and comprehensiveness, and organization. OpenEvidence uniquely matched the control in literature support, with 24 (100%) complete and accurate citations. Grok was the most speech-efficient; Gemini was the most verbose. The readability for each LLM was, on average, lower quality than required for scientific technical writing.
Conclusion
This study shows that each AI model does not compare to a human expert's ability to generate scientific technical writing in a subspecialty of ophthalmology. Only OpenEvidence demonstrated citation accuracy meeting review article standards; however, this platform did not meet standards in other areas. The study concludes that current unpaid AI platforms cannot replace human expertise in creating high-level scientific literature in ophthalmology.
Related
- Generative artificial intelligence in the preparation of scientific medical articles on surgery: principles of responsible use and current requirements of editorial organisations
- Neuroimaging in neuro-ophthalmologic emergencies with acute visual loss
- Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study
- Female leadership across ophthalmology subspecialties: a cross-sectional study of academic leadership roles
- Clinical predictions using artificial intelligence in neurosurgery and its subspecialties: a systematic review and meta-analysis
BibTeX
@article{jones2026comparative,
title = {{Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties}},
author = {Brooklyn Z Jones and Jeffrey R Olsen and Sabrina M Miller and Eduardo Herrera and Danielle R Okelberry and Mina M Sitto and Traeson Brandenburg and Phillip C Hoopes and Majid Moshirfar},
journal = {Cureus},
year = {2026},
month = oct,
doi = {10.7759/cureus.117869},
url = {https://doi.org/10.7759/cureus.117869},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties
AU - Brooklyn Z Jones
AU - Jeffrey R Olsen
AU - Sabrina M Miller
AU - Eduardo Herrera
AU - Danielle R Okelberry
AU - Mina M Sitto
AU - Traeson Brandenburg
AU - Phillip C Hoopes
AU - Majid Moshirfar
PY - 2026
DA - 2026/10/13
JO - Cureus
DO - 10.7759/cureus.117869
UR - https://doi.org/10.7759/cureus.117869
AB - Purpose As the adoption of large language models (LLMs) accelerates in healthcare, the evaluation of their accuracy must accelerate in tandem. This study compares six LLMs' ability to generate accurate, thorough, and high-quality scientific technical writing in seven areas compared to a human-written control. Methods Four prompts were created in ophthalmic subspecialties and presented to ChatGPT (4.0), Gemini (2.5 Pro), DeepSeek (DeepSeek-Prover-V2-671B), Grok (3.0), Microsoft Copilot, and OpenEvidence (2.0). Each model's performance was analyzed based on seven factors: medical accuracy, thoroughness and comprehensiveness, organization, literature support, speech efficiency, citation classification, and readability. Data analysis was conducted in Excel, graphed, and compared to a control. Results The control texts outperformed each of the artificial intelligence (AI) models in medical accuracy, thoroughness and comprehensiveness, and organization. OpenEvidence uniquely matched the control in literature support, with 24 (100%) complete and accurate citations. Grok was the most speech-efficient; Gemini was the most verbose. The readability for each LLM was, on average, lower quality than required for scientific technical writing. Conclusion This study shows that each AI model does not compare to a human expert's ability to generate scientific technical writing in a subspecialty of ophthalmology. Only OpenEvidence demonstrated citation accuracy meeting review article standards; however, this platform did not meet standards in other areas. The study concludes that current unpaid AI platforms cannot replace human expertise in creating high-level scientific literature in ophthalmology.
SN - 2168-8184
ER -