Evaluating GPT-4o Model Proficiency and Clinical Reasoning for Antimicrobial Stewardship in Dentistry
- 1Faculty of Dental Medicine and Oral Health Sciences, McGill University, Montreal, Quebec, Canada
Abstract
ObjectivesDentists prescribe approximately one in ten antibiotics worldwide, yet antimicrobial stewardship (AMS) remains underemphasized in dental education. Large language models (LLMs) may support AMS training, but their proficiency and clinical reasoning in this context remain unclear. We evaluated GPT-4os accuracy and clinical reasoning on dental antibiotic prescribing questions, stratified by question difficulty.
MethodsWe assembled 125 multiple-choice questions on dental antibiotic prescribing from eight peer-reviewed studies (2017-2023). GPT-4o answered each question and generated a clinical justification. Accuracy was assessed against source-study answer keys and examined across difficulty quartiles. Justifications were evaluated using an adapted 12-axis human-evaluation framework assessing scientific consensus, extent and likelihood of harm, inappropriate and missing content, bias, and both correct and incorrect comprehension, retrieval, and reasoning. Prophylaxis-specific questions were analysed separately.
ResultsGPT-4o correctly answered 72% of questions. Accuracy remained relatively stable across difficulty quartiles (78%, 78%, 65%, 70%). Experts rated 95.4% of justifications positively across the 12 axes. Comprehension, retrieval, and reasoning each exceeded 96.2% positive ratings. Missing content was the main weakness (7.8%), and 7.1% of justifications showed a moderate-to-severe potential for harm. Performance on prophylaxis-specific questions (98.1%) exceeded non-prophylaxis questions (93.0%).
ConclusionsGPT-4o demonstrated moderate-to-high proficiency and clinically defensible reasoning in dental antibiotic prescribing questions. However, residual risks indicate that it is not suitable for unsupervised clinical use but shows potential as a supervised AMS educational tool.
Related
- Application of DASC and AUSC metrics in evaluating antimicrobial stewardship interventions in critically ill patients
- Aligning Antimicrobial Stewardship and Infection Prevention Through Repeated Assessment and Feedback: A Multisite Quality Improvement Initiative in Viet Nam
- Azithromycin in crisis: antimicrobial resistance, diagnostics gaps, and vaccine-linked stewardship for enteric infections in low- and middle-income countries
- Implementation of antimicrobial stewardship programmes in hospitals of the Brazilian public health system: PeGASUS study protocol
- Procalcitonin and Presepsin in Gram-Negative Sepsis: Diagnostic, Prognostic, and Antimicrobial Stewardship Perspectives
BibTeX
@misc{dick2026evaluating,
title = {{Evaluating GPT-4o Model Proficiency and Clinical Reasoning for Antimicrobial Stewardship in Dentistry}},
author = {M. Dick and S. Madathil and A. Patel and H. S. Kapoor and M. Sharma and Z. D'Souza and S. Hameed and M. Abu-Samak and A. Najirad and D. Dwairi and O. Radaideh and B. Nicolau},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.01.26361980},
url = {https://www.medrxiv.org/content/10.64898/2026.09.01.26361980v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Evaluating GPT-4o Model Proficiency and Clinical Reasoning for Antimicrobial Stewardship in Dentistry
AU - M. Dick
AU - S. Madathil
AU - A. Patel
AU - H. S. Kapoor
AU - M. Sharma
AU - Z. D'Souza
AU - S. Hameed
AU - M. Abu-Samak
AU - A. Najirad
AU - D. Dwairi
AU - O. Radaideh
AU - B. Nicolau
PY - 2026
DA - 2026/09/01
PB - medRxiv
DO - 10.64898/2026.09.01.26361980
UR - https://www.medrxiv.org/content/10.64898/2026.09.01.26361980v1
AB - ObjectivesDentists prescribe approximately one in ten antibiotics worldwide, yet antimicrobial stewardship (AMS) remains underemphasized in dental education. Large language models (LLMs) may support AMS training, but their proficiency and clinical reasoning in this context remain unclear. We evaluated GPT-4os accuracy and clinical reasoning on dental antibiotic prescribing questions, stratified by question difficulty. MethodsWe assembled 125 multiple-choice questions on dental antibiotic prescribing from eight peer-reviewed studies (2017-2023). GPT-4o answered each question and generated a clinical justification. Accuracy was assessed against source-study answer keys and examined across difficulty quartiles. Justifications were evaluated using an adapted 12-axis human-evaluation framework assessing scientific consensus, extent and likelihood of harm, inappropriate and missing content, bias, and both correct and incorrect comprehension, retrieval, and reasoning. Prophylaxis-specific questions were analysed separately. ResultsGPT-4o correctly answered 72% of questions. Accuracy remained relatively stable across difficulty quartiles (78%, 78%, 65%, 70%). Experts rated 95.4% of justifications positively across the 12 axes. Comprehension, retrieval, and reasoning each exceeded 96.2% positive ratings. Missing content was the main weakness (7.8%), and 7.1% of justifications showed a moderate-to-severe potential for harm. Performance on prophylaxis-specific questions (98.1%) exceeded non-prophylaxis questions (93.0%). ConclusionsGPT-4o demonstrated moderate-to-high proficiency and clinically defensible reasoning in dental antibiotic prescribing questions. However, residual risks indicate that it is not suitable for unsupervised clinical use but shows potential as a supervised AMS educational tool.
ER -