How reliable are large language models in pediatric dentistry? Performance of seven artificial intelligence models on dental specialty examination questions (2012–2024)
- 1Department of Pediatric Dentistry, Faculty of Dentistry, Erciyes University
- 2Department of Pediatric Dentistry, Faculty of Dentistry, Nuh Naci Yazgan University
Abstract
Background Large language models (LLMs) are increasingly being used in dental education; however, their reliability in the field of pediatric dentistry remains unclear. This study aimed to compare the accuracy rates of seven different LLMs in answering pediatric dentistry questions from the Turkish Dental Specialty Examination (DSE). Methods A cross-sectional evaluation was conducted on a total of 174 multiple-choice pediatric dentistry questions derived from 18 DSE administered between 2012 and 2024. The questions were categorized into 13 subdomains and further classified as clinical/case-based or theoretical/knowledge-based. The performance of seven different LLMs (ChatGPT-4.1 Pro, ChatGPT-5 Pro, Claude Pro, Copilot Pro, DeepSeek-R1, Gemini Advanced 2.5 Experimental, and Perplexity Pro) was assessed. Accuracy rates were compared across models, subdomains, and question types, and factors predicting correct responses were identified using multivariate logistic regression analysis. Results Accuracy rates differed significantly among the models (p = 0.019). The highest accuracy was observed with Gemini Advanced 2.5 Experimental (93.7%), whereas the lowest was with Claude Pro (82.8%). Accuracy rates also varied significantly across subdomains (p < 0.001), with lower performance observed in Restorative treatments/materials (75.5%) and Fluoride/preventive applications (78.6%). Clinical questions were answered with higher accuracy than theoretical questions (93.7% vs. 86.2%). In the regression analysis, the Gemini Advanced 2.5 Experimental model and clinical question type were identified as independent predictors of correct responses compared with ChatGPT-5 Pro. Conclusion LLMs demonstrated high performance in answering pediatric dentistry questions derived from the Turkish DSE; however, these findings should be interpreted in light of the standardized, examination-based nature of the assessment. LLMs should be used as complementary tools in pediatric dentistry education and similar domain-specific educational contexts, rather than as substitutes for expert clinical judgment.
Related
- Artificial intelligence in pediatric dentistry: reliability and readability analysis of AAPD FAQ responses
- Artificial Intelligence Applications in Pediatric Dentistry: A Systematic Scoping Review With Visual Evidence Mapping.
- Psychometric characterization of human and artificial intelligence performance on cardiology residency in-service examination items
- Artificial Intelligence in Contemporary Dentistry: Applications Across Dental Specialties and Future Challenges.
- Toward integrated multimodal artificial intelligence in bone marrow examination: current state and open challenges
BibTeX
@article{gumus2026reliable,
title = {{How reliable are large language models in pediatric dentistry? Performance of seven artificial intelligence models on dental specialty examination questions (2012–2024)}},
author = {Hüsniye Gümüş and Gülce Öztürk and Aybüke Bahadır Sezer},
journal = {BMC Medical Education},
year = {2026},
month = aug,
doi = {10.1186/s12909-026-10190-1},
url = {https://doi.org/10.1186/s12909-026-10190-1},
note = {CC BY},
}
RIS
TY - JOUR
TI - How reliable are large language models in pediatric dentistry? Performance of seven artificial intelligence models on dental specialty examination questions (2012–2024)
AU - Hüsniye Gümüş
AU - Gülce Öztürk
AU - Aybüke Bahadır Sezer
PY - 2026
DA - 2026/08/01
JO - BMC Medical Education
DO - 10.1186/s12909-026-10190-1
UR - https://doi.org/10.1186/s12909-026-10190-1
AB - Background Large language models (LLMs) are increasingly being used in dental education; however, their reliability in the field of pediatric dentistry remains unclear. This study aimed to compare the accuracy rates of seven different LLMs in answering pediatric dentistry questions from the Turkish Dental Specialty Examination (DSE). Methods A cross-sectional evaluation was conducted on a total of 174 multiple-choice pediatric dentistry questions derived from 18 DSE administered between 2012 and 2024. The questions were categorized into 13 subdomains and further classified as clinical/case-based or theoretical/knowledge-based. The performance of seven different LLMs (ChatGPT-4.1 Pro, ChatGPT-5 Pro, Claude Pro, Copilot Pro, DeepSeek-R1, Gemini Advanced 2.5 Experimental, and Perplexity Pro) was assessed. Accuracy rates were compared across models, subdomains, and question types, and factors predicting correct responses were identified using multivariate logistic regression analysis. Results Accuracy rates differed significantly among the models (p = 0.019). The highest accuracy was observed with Gemini Advanced 2.5 Experimental (93.7%), whereas the lowest was with Claude Pro (82.8%). Accuracy rates also varied significantly across subdomains (p < 0.001), with lower performance observed in Restorative treatments/materials (75.5%) and Fluoride/preventive applications (78.6%). Clinical questions were answered with higher accuracy than theoretical questions (93.7% vs. 86.2%). In the regression analysis, the Gemini Advanced 2.5 Experimental model and clinical question type were identified as independent predictors of correct responses compared with ChatGPT-5 Pro. Conclusion LLMs demonstrated high performance in answering pediatric dentistry questions derived from the Turkish DSE; however, these findings should be interpreted in light of the standardized, examination-based nature of the assessment. LLMs should be used as complementary tools in pediatric dentistry education and similar domain-specific educational contexts, rather than as substitutes for expert clinical judgment.
SN - 1472-6920
ER -