{
  "abstract": "Objective\n\nTo compare expert-rated quality and safety of responses from four contemporary large language models (LLMs)-ChatGPT, Gemini, Claude, and Grok-on expert-level anterior segment ophthalmology questions.Design\n\nCross-sectional, masked, randomized-display-order, multievaluator comparative study.Participants\n\nEleven fellowship-trained cornea and anterior segment subspecialists.Methods\n\nForty-five expert-level questions spanning six anterior segment domains were administered to each LLM under zero-shot conditions in a single prespecified session (January 19, 2026), with memory, browsing, and retrieval disabled. Responses were masked to model identity and independently rated on five 5-point Likert metrics (expert-rated accuracy, completeness/clinical utility, clarity/readability, overall safety, and potential for harm). No specialist-authored reference answer key was used; all metrics index expert-rated perception rather than verified clinical correctness.Main outcome measures\n\nMedian [interquartile range], mean ± standard deviation, favorable-rating proportion (≤2) with 95% bootstrap confidence interval, Friedman omnibus with Kendall's W, and Bonferroni-adjusted pairwise Wilcoxon tests with rank-biserial correlation. Prespecified sensitivity analyses included length-adjusted mixed models and a masking-integrity diagnostic; post hoc analyses added question-level severe-safety and style-restricted comparisons.Results\n\nAll 9900 ratings (45 questions × 4 LLMs × 5 metrics × 11 evaluators) were complete. Differences were significant for all three quality metrics (Friedman P < 0.001; Kendall's W = 0.44-0.54); Claude and Grok were each rated more favorably than ChatGPT and Gemini (all adjusted P < 0.001; |r_rb| = 0.73-0.98). No statistically significant between-model difference was detected in mean safety scores. Post hoc, more questions elicited ≥ 1 severe overall-safety rating for Claude (35.6%) and Grok (33.3%) than for ChatGPT (6.7%) or Gemini (11.1%) (Cochran Q = 23.4, P < 0.001), persisting when two-evaluator concurrence was required. Ordinal Krippendorff's α was 0.021-0.168; the 11-evaluator mean used as the primary analytic unit showed moderate reliability (intraclass correlation coefficient [ICC(2,k)] = 0.58-0.69 for quality metrics).Conclusions\n\nClaude and Grok received more favorable expert-rated quality than ChatGPT and Gemini. Post hoc, a larger severe-safety-rating tail was identified for these two models despite no significant difference in mean safety; this signal is hypothesis-generating. The findings reflect expert preference, not validated clinical correctness, at one time point.Financial disclosures\n\nThe authors have no proprietary or commercial interest in any materials discussed in this article.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Ophthalmology, Eunpyeong St. Mary's Hospital, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea."
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 3,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Soonwon Yang"
    },
    {
      "affs": [
        0
      ],
      "name": "Hyoji Han"
    },
    {
      "affs": [
        0
      ],
      "name": "Hyun Soo Lee"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/6watn2s6dn416.bib",
  "cited_by": 0,
  "doi": "10.1016/j.xops.2026.101398",
  "fulltext_url": "https://europepmc.org/articles/PMC13677228",
  "id": "6watn2s6dn416",
  "issns": [
    "2666-9145"
  ],
  "landing_url": "https://europepmc.org/article/MED/42894879",
  "language": "eng",
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7208757176",
  "pdf_url": "https://europepmc.org/articles/PMC13677228?pdf=render",
  "pmcid": "PMC13677228",
  "pmid": "42894879",
  "published": "2026-09-04",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/6watn2s6dn416.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:19:41Z",
      "native_id": "MED:42894879",
      "source": "europepmc",
      "url": "https://europepmc.org/article/MED/42894879"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Evaluator-Masked Assessment of 4 Consumer-Facing Large Language Models on Expert Anterior Segment Ophthalmology Questions.",
  "title_text": "Evaluator-Masked Assessment of 4 Consumer-Facing Large Language Models on Expert Anterior Segment Ophthalmology Questions.",
  "type": "peer-reviewed",
  "updated_at": "2026-10-30T12:19:41Z",
  "url": "https://research.draconitic.com/paper/6watn2s6dn416",
  "venue": "Ophthalmology science",
  "versions": []
}