Skip to content
DraconiticResearch
Search Sources API
226,742 studies31 sourcesUpdated 36 min ago
Peer-reviewedCC BY 4.0Cureus

Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties

9 affiliations
  1. 1Department of Ophthalmology, Midwestern University College of Osteopathic Medicine, Phoenix, USA
  2. 2Department of Ophthalmology, Creighton University College of Medicine, Phoenix, USA
  3. 3Department of Ophthalmology, Michigan State University College of Osteopathic Medicine, Warren, USA
  4. 4Department of Ophthalmology, Albany Medical College, New York, USA
  5. 5Department of Ophthalmology, Rocky Vista University College of Osteopathic Medicine, St. George, USA
  6. 6Department of Ophthalmology, Hoopes Moshirfar Research Center, Draper, USA
  7. 7Department of Ophthalmology, Hoopes Vision, Draper, USA
  8. 8John A. Moran Eye Center, University of Utah School of Medicine, Salt Lake City, USA
  9. 9Eye Banking and Corneal Transplantation, Utah Lions Eye Bank, Murray, USA

Abstract

Purpose

As the adoption of large language models (LLMs) accelerates in healthcare, the evaluation of their accuracy must accelerate in tandem. This study compares six LLMs' ability to generate accurate, thorough, and high-quality scientific technical writing in seven areas compared to a human-written control.

Methods

Four prompts were created in ophthalmic subspecialties and presented to ChatGPT (4.0), Gemini (2.5 Pro), DeepSeek (DeepSeek-Prover-V2-671B), Grok (3.0), Microsoft Copilot, and OpenEvidence (2.0). Each model's performance was analyzed based on seven factors: medical accuracy, thoroughness and comprehensiveness, organization, literature support, speech efficiency, citation classification, and readability. Data analysis was conducted in Excel, graphed, and compared to a control.

Results

The control texts outperformed each of the artificial intelligence (AI) models in medical accuracy, thoroughness and comprehensiveness, and organization. OpenEvidence uniquely matched the control in literature support, with 24 (100%) complete and accurate citations. Grok was the most speech-efficient; Gemini was the most verbose. The readability for each LLM was, on average, lower quality than required for scientific technical writing.

Conclusion

This study shows that each AI model does not compare to a human expert's ability to generate scientific technical writing in a subspecialty of ophthalmology. Only OpenEvidence demonstrated citation accuracy meeting review article standards; however, this platform did not meet standards in other areas. The study concludes that current unpaid AI platforms cannot replace human expertise in creating high-level scientific literature in ophthalmology.

Related

  1. Generative artificial intelligence in the preparation of scientific medical articles on surgery: principles of responsible use and current requirements of editorial organisations
    O. Yu. Usenko, I. A. Kryvoruchko, S. A. Andreieshchev +1
    Український журнал клінічної хірургіїPeer-reviewedCC BYFull textEpidemiology & public healthDOAJ
  2. Neuroimaging in neuro-ophthalmologic emergencies with acute visual loss
    Wei-Ling Lin, Rong-Kung Tsai
    Taiwan Journal of OphthalmologyPeer-reviewedCC BY-NC-NDFull textGeneral medicineDOAJ
  3. Reliability, Readability, and Safety of Generative AI Chatbots for Osteoporosis Health Counseling: A Multidimensional Cross‐Sectional Comparative Study
    Huimin Xu, Liang Guo, Li Shen +3
    Health science reportsPeer-reviewedCC BY-NC-NDFull textGeneral medicineEurope PMC
  4. Female leadership across ophthalmology subspecialties: a cross-sectional study of academic leadership roles
    Ana M. Montoya Zuluaga, Meghan K. Berkenstock
    BMC Medical EducationPeer-reviewedCC BY-NC-ND 4.0Full textGeneral medicinePMC
  5. Clinical predictions using artificial intelligence in neurosurgery and its subspecialties: a systematic review and meta-analysis
    Prajwal Ghimire, Aswin Chari, Jonathan Shapey +1
    arXivPreprintCC BY 4.0Full textPhysiologyarXiv
BibTeX
@article{jones2026comparative,
  title = {{Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties}},
  author = {Brooklyn Z Jones and Jeffrey R Olsen and Sabrina M Miller and Eduardo Herrera and Danielle R Okelberry and Mina M Sitto and Traeson Brandenburg and Phillip C Hoopes and Majid Moshirfar},
  journal = {Cureus},
  year = {2026},
  month = oct,
  doi = {10.7759/cureus.117869},
  url = {https://doi.org/10.7759/cureus.117869},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Comparative Analysis of Six Large Language Models' Generative Ability for Scientific Technical Writing in Ophthalmologic Subspecialties
AU  - Brooklyn Z Jones
AU  - Jeffrey R Olsen
AU  - Sabrina M Miller
AU  - Eduardo Herrera
AU  - Danielle R Okelberry
AU  - Mina M Sitto
AU  - Traeson Brandenburg
AU  - Phillip C Hoopes
AU  - Majid Moshirfar
PY  - 2026
DA  - 2026/10/13
JO  - Cureus
DO  - 10.7759/cureus.117869
UR  - https://doi.org/10.7759/cureus.117869
AB  - Purpose  As the adoption of large language models (LLMs) accelerates in healthcare, the evaluation of their accuracy must accelerate in tandem. This study compares six LLMs' ability to generate accurate, thorough, and high-quality scientific technical writing in seven areas compared to a human-written control.  Methods  Four prompts were created in ophthalmic subspecialties and presented to ChatGPT (4.0), Gemini (2.5 Pro), DeepSeek (DeepSeek-Prover-V2-671B), Grok (3.0), Microsoft Copilot, and OpenEvidence (2.0). Each model's performance was analyzed based on seven factors: medical accuracy, thoroughness and comprehensiveness, organization, literature support, speech efficiency, citation classification, and readability. Data analysis was conducted in Excel, graphed, and compared to a control.  Results  The control texts outperformed each of the artificial intelligence (AI) models in medical accuracy, thoroughness and comprehensiveness, and organization. OpenEvidence uniquely matched the control in literature support, with 24 (100%) complete and accurate citations. Grok was the most speech-efficient; Gemini was the most verbose. The readability for each LLM was, on average, lower quality than required for scientific technical writing.  Conclusion  This study shows that each AI model does not compare to a human expert's ability to generate scientific technical writing in a subspecialty of ophthalmology. Only OpenEvidence demonstrated citation accuracy meeting review article standards; however, this platform did not meet standards in other areas. The study concludes that current unpaid AI platforms cannot replace human expertise in creating high-level scientific literature in ophthalmology.
SN  - 2168-8184
ER  -