AI chatbots showed higher response accuracy than dentists, medical doctors, and the general public on radiation safety questions
Abstract
Radiation is encountered in daily life through natural and artificial sources as well as in medical diagnostic and therapeutic applications, and its exposure may lead to adverse health effects. This study aimed to assess radiation-related knowledge among dentists, medical doctors, and the general public and to compare their response accuracy with that of AI-based chatbot systems. A 25-item multiple-choice questionnaire assessing radiation risks and radiation safety was administered to 344 individuals, including dentists, medical doctors, and members of the general public. The same questions were entered into seven selected AI-based chatbot systems (Claude, DeepSeek, Microsoft Copilot, Gemini, Perplexity AI, ChatGPT-5, and ChatGPT-4o). The response accuracy of the chatbots was compared with that of the human participants. In addition, the consistency of their performance across different users and time periods was comparatively analyzed. AI-based systems generally demonstrated higher accuracy rates compared to human participants. Accuracy rates ranged from 80 to 90.67% among AI-based systems, compared with 32–60% among human participants. Among the evaluated chatbot systems, Microsoft Copilot achieved the highest performance, while Perplexity AI exhibited the lowest. Among the human groups, dentists and medical doctors performed significantly better than the general public, with no significant difference between the two healthcare professional groups. Performance remained stable across morning, midday, and evening assessments for most AI systems; however, ChatGPT-5 showed a significant temporal difference ( p < 0.001). AI-based systems achieved higher response accuracy than human participants under the evaluation conditions used in this study. The overall performance of the human participant groups indicates room for improvement in radiation safety education. While AI-based tools may serve as valuable educational resources, their performance should not be interpreted as evidence of superior clinical judgment or decision-making ability.
Related
- Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
- Multidimensional evaluation of generative AI chatbots for public consultation on sudden cardiac death: safety, accuracy, empathy, reliability, quality, and readability across six models
- Performance of AI chatbots in responding to patients' frequently asked questions about laminate veneer restorations.
- The use of different artificial intelligence-powered chatbots to answer questions related to dental implants
- How well do large language models answer postoperative lumbar fusion questions? A blinded comparative analysis of accuracy, quality, readability, and safety-related content
BibTeX
@article{karslıoglu2026chatbots,
title = {{AI chatbots showed higher response accuracy than dentists, medical doctors, and the general public on radiation safety questions}},
author = {Hazal karslıoğlu and Fatıma Sümeyra Çetin and Halil Altun},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-77832-8},
url = {https://www.nature.com/articles/s41598-026-77832-8},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - AI chatbots showed higher response accuracy than dentists, medical doctors, and the general public on radiation safety questions
AU - Hazal karslıoğlu
AU - Fatıma Sümeyra Çetin
AU - Halil Altun
PY - 2026
DA - 2026/10/26
JO - Scientific Reports
DO - 10.1038/s41598-026-77832-8
UR - https://www.nature.com/articles/s41598-026-77832-8
AB - Radiation is encountered in daily life through natural and artificial sources as well as in medical diagnostic and therapeutic applications, and its exposure may lead to adverse health effects. This study aimed to assess radiation-related knowledge among dentists, medical doctors, and the general public and to compare their response accuracy with that of AI-based chatbot systems. A 25-item multiple-choice questionnaire assessing radiation risks and radiation safety was administered to 344 individuals, including dentists, medical doctors, and members of the general public. The same questions were entered into seven selected AI-based chatbot systems (Claude, DeepSeek, Microsoft Copilot, Gemini, Perplexity AI, ChatGPT-5, and ChatGPT-4o). The response accuracy of the chatbots was compared with that of the human participants. In addition, the consistency of their performance across different users and time periods was comparatively analyzed. AI-based systems generally demonstrated higher accuracy rates compared to human participants. Accuracy rates ranged from 80 to 90.67% among AI-based systems, compared with 32–60% among human participants. Among the evaluated chatbot systems, Microsoft Copilot achieved the highest performance, while Perplexity AI exhibited the lowest. Among the human groups, dentists and medical doctors performed significantly better than the general public, with no significant difference between the two healthcare professional groups. Performance remained stable across morning, midday, and evening assessments for most AI systems; however, ChatGPT-5 showed a significant temporal difference ( p < 0.001). AI-based systems achieved higher response accuracy than human participants under the evaluation conditions used in this study. The overall performance of the human participant groups indicates room for improvement in radiation safety education. While AI-based tools may serve as valuable educational resources, their performance should not be interpreted as evidence of superior clinical judgment or decision-making ability.
SN - 2045-2322
ER -