{
  "abstract": "PurposeTo compare the performance of three large language models (LLMs)--ChatGPT-o1, Grok 3, and Gemini 2.0 Pro--on the American College of Prosthodontists National Prosthodontic Resident Examination (NPRE).\n\nMaterials and MethodsOfficial NPRE examinations from years 2015-2024 were retrieved from the American College of Prosthodontists website and used to prompt each LLM. Paid access tiers and advanced reasoning were enabled. Responses from LLMs were recorded and compared against the official answer keys for accuracy, variability across years, and consistency of responses. Duplicate questions were retained for the primary analysis and removed in a sensitivity analysis. Accuracy (percent correct) was compared using generalized estimating equations binary logistic regression (primary); Cochrans Q with Bonferroni-adjusted McNemar tests (sensitivity). Year-to-year variability (SD of annual accuracy) was compared using weighted Levene tests. Consistency across duplicate items was quantified as the percentage of perfect matches and compared using Cochrans Q with Bonferroni-adjusted McNemar tests.\n\nResultsAcross 1,487 questions, overall accuracy differed among models (global P < 0.001). ChatGPT-o1 achieved 78.4% correct, Grok 3 76.4%, and Gemini 2.0 Pro 73.2%; ChatGPT-o1 and Grok 3s accuracy significantly exceeded Gemini 2.0 Pro (both P [≤] 0.016), with no significant difference between ChatGPT-o1 and Grok 3 (P = 0.099). After removing duplicates (n = 1,200), results were similar: ChatGPT-o1, Grok 3, and Gemini 2.0 Pro were 77.1%, 76.6%, and 71.5% correct, respectively; ChatGPT-o1 and Grok 3s accuracy was significantly higher than Gemini 2.0 Pro (both P < 0.001). Variability across years did not differ significantly (P [≥] 0.097). Grok 3 showed significantly higher perfect-match consistency (79.2%) than ChatGPT-o1 (69.4%) and Gemini 2.0 Pro (64.5%) (P [≤] 0.010).\n\nConclusionsContemporary LLMs showed 70-80% accuracy on postgraduate prosthodontic multiple-choice content. While promising for educational support, residual error and inconsistency warrant human oversight and transparent methods reporting before curricular adoption.",
  "abstract_shown": true,
  "affiliations": [
    "Tufts University School of Dental Medicine"
  ],
  "article_type": "PUBLISHAHEADOFPRINT",
  "arxiv": null,
  "author_total": 5,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Y.-w. Chen"
    },
    {
      "name": "V. Bhunyanaphakul"
    },
    {
      "name": "P. Kosago"
    },
    {
      "name": "M. Finkelman"
    },
    {
      "name": "D. J. Lee"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/c8x8p07aac88w.bib",
  "cited_by": 0,
  "doi": "10.64898/2026.10.19.26365937",
  "fulltext_url": "https://www.medrxiv.org/content/10.64898/2026.10.19.26365937v1.full",
  "id": "c8x8p07aac88w",
  "issns": [],
  "landing_url": "https://www.medrxiv.org/content/10.64898/2026.10.19.26365937v1",
  "language": null,
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "medrxiv",
      "dentistry and oral medicine"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7223114426",
  "pdf_url": "https://www.medrxiv.org/content/10.64898/2026.10.19.26365937v1.full.pdf",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-19",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/c8x8p07aac88w.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:21:22Z",
      "native_id": "10.64898/2026.10.19.26365937",
      "source": "medrxiv",
      "url": "https://www.medrxiv.org/content/10.64898/2026.10.19.26365937v1"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Performance of Contemporary Large Language Models on the National Prosthodontic Resident Examination: A Multi-Year Comparative Study",
  "title_text": "Performance of Contemporary Large Language Models on the National Prosthodontic Resident Examination: A Multi-Year Comparative Study",
  "type": "preprint",
  "updated_at": "2026-10-30T12:21:22Z",
  "url": "https://research.draconitic.com/paper/c8x8p07aac88w",
  "venue": "medRxiv",
  "versions": []
}