{
  "abstract": "Background\n\nLarge language models (LLMs) are increasingly used as study aids in dental education, yet their performance may vary by model, discipline, and question format. This study benchmarked LLM-based systems using Turkish Dentistry Specialization Examination (DUS) Clinical Sciences questions.Methods\n\nA public dataset of DUS Clinical Sciences multiple-choice questions from 2012 to 2021 (n = 1,027; excluding officially canceled items) was compiled from the national examination authority. Each model was evaluated once per question: ChatGPT-4o, ChatGPT-o4-mini-high, ChatGPT-5 Thinking, Gemini-2.5 Pro, Gemini-2.0 Flash, Claude Sonnet 4, Copilot, and DeepSeek-v3.1. Accuracy was scored against the official answer key. Questions were categorized as standard, K-type, or visual. Paired question-level analyses and 95% confidence intervals were used for accuracy outcomes; response time was recorded on the same endodontics question set.Results\n\nOverall accuracy exceeded 80% for all models, and paired analysis confirmed significant differences among models (Cochran's Q(7) = 247.27; p < 0.001). ChatGPT-5 Thinking achieved the highest accuracy (94.26%), followed by Gemini-2.5 Pro (92.50%); DeepSeek-v3.1 had the lowest accuracy (80.92%). Accuracy differed markedly by question format (chi-square(2) = 391.690; p < 0.001): 89.61% for standard, 75.71% for K-type, and 55.56% for visual items. Response times differed across models (Kruskal-Wallis H(7) = 649.243; p < 0.001): Claude Sonnet 4 and Copilot were fastest, while Gemini-2.5 Pro was slowest.Conclusions\n\nPublicly available LLM-based systems can perform strongly on DUS-level questions, but performance is sensitive to model choice, discipline, question format, and time cost. Outputs, especially for visual and K-type items, should be used as supportive educational material and verified with reliable sources.",
  "abstract_shown": true,
  "affiliations": [
    "Department of Endodontics, Faculty of Dentistry, Akdeniz University, Antalya, Türkiye. ozgesahin3399@gmail.com.",
    "Department of Endodontics, Faculty of Dentistry, Akdeniz University, Antalya, Türkiye."
  ],
  "article_type": "research article",
  "arxiv": null,
  "author_total": 2,
  "authors": [
    {
      "affs": [
        0
      ],
      "name": "Nisa Özge Şahin",
      "orcid": "0009-0002-6520-3063"
    },
    {
      "affs": [
        1
      ],
      "name": "Alper Kuştarcı",
      "orcid": "0000-0002-4942-3739"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/bqxjneqsjmaa8.bib",
  "cited_by": 0,
  "doi": "10.1186/s12903-026-09323-1",
  "fulltext_url": "https://europepmc.org/articles/PMC13749691",
  "id": "bqxjneqsjmaa8",
  "issns": [
    "1472-6831"
  ],
  "landing_url": "https://europepmc.org/article/MED/43009941",
  "language": "eng",
  "licence": "CC-BY-NC-ND",
  "licence_label": "CC BY-NC-ND",
  "licence_source": "article",
  "licence_url": "https://creativecommons.org/licenses/by-nc-nd/4.0/",
  "metadata_licence": null,
  "native_subjects": [
    [
      "heading",
      "Research"
    ],
    [
      "openalex",
      "2718|Artificial Intelligence in Healthcare and Education"
    ]
  ],
  "openalex": "W7202273854",
  "pdf_url": "https://europepmc.org/articles/PMC13749691?pdf=render",
  "pmcid": "PMC13749691",
  "pmid": "43009941",
  "published": "2026-08-12",
  "publisher": "BMC",
  "ris": "https://research.draconitic.com/paper/bqxjneqsjmaa8.ris",
  "sources": [
    {
      "first_seen": "2026-11-12T15:22:43Z",
      "native_id": "MED:43009941",
      "source": "europepmc",
      "url": "https://europepmc.org/article/MED/43009941"
    },
    {
      "first_seen": "2026-11-14T12:22:24Z",
      "native_id": "PMC13749691",
      "source": "pmc",
      "url": "https://doi.org/10.1186/s12903-026-09323-1"
    }
  ],
  "subjects": [
    {
      "field": "medicine",
      "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education",
      "subfield": "epidemiology-public-health"
    }
  ],
  "title": "Benchmarking large language models on Turkish dental specialty examination questions: effects of model, discipline, and question format.",
  "title_text": "Benchmarking large language models on Turkish dental specialty examination questions: effects of model, discipline, and question format.",
  "type": "peer-reviewed",
  "updated_at": "2026-11-14T12:22:24Z",
  "url": "https://research.draconitic.com/paper/bqxjneqsjmaa8",
  "venue": "BMC oral health",
  "versions": []
}