Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation
6 affiliations
- 1Department of Pediatric and Preventive Dentistry, Luxmi Bai Institute of Dental Sciences and Hospital, Patiala, India
- 2Department of Oral Medicine and Oral Pathology, Griffith School of Medicine and Dentistry, Southport, QLD, Australia
- 3Department of Periodontics, Luxmi Bai Institute of Dental Sciences and Hospital, Patiala, India
- 4Department of Oral Medicine and Radiology, Dashmesh Institute of Research and Dental Sciences, Faridkot, India
- 5Department of Oral Medicine & Radiology, Dr Ziauddin Ahmad Dental College & Hospital, Aligarh Muslim University Campus, Aligarh, India
- 6Dentistry and Oral Health, La Trobe University, Bendigo, VIC, Australia
Abstract
Background: Consumer-facing artificial intelligence platforms are increasingly used to obtain health information, but the quality of their responses to dental radiology questions requires evaluation. This study compared expert ratings of single responses generated by three selected consumer-facing AI platform configurations to common patient-facing dental radiology questions.
Methods: Twenty expert-reviewed English-language questions were submitted using one standardized prompt to the investigator-recorded, user-visible ChatGPT-5.5 configuration with Thinking enabled, Gemini 3 Flash free configuration, and Microsoft Copilot 2026 free configuration during late April 2026. One response was collected from each platform for each question, producing 60 responses. Three Oral and Maxillofacial Radiologists independently rated coded responses with platform labels concealed, assessing accuracy, relevance, clarity, completeness, safety, triage appropriateness and misinformation. The 20 questions were treated as the primary paired experimental units. Evaluator-averaged scores were compared across platform configurations using Friedman tests followed by question-level exact paired comparisons, while individual evaluator ratings were used descriptively and for agreement assessment.
Results: The selected ChatGPT configuration received more favourable ratings than Gemini and Copilot for accuracy, relevance, safety and misinformation. It also received more favourable clarity and completeness ratings than Copilot. Gemini received more favourable question-level ratings than Copilot for clarity and completeness, but not for relevance. No significant differences were observed between ChatGPT and Gemini for clarity or completeness. Triage was applicable only to Question 20 and was reported descriptively. Considerable ceiling effects and tied ratings were observed, including 100% maximum ratings for ChatGPT accuracy and safety.
Conclusion: Under the specified testing conditions, expert ratings differed across the single outputs generated by the selected platform configurations. The findings are limited by unequal platform configurations, one generation per question, unavailable historical interface metadata, a relatively homogeneous informational question set, ceiling effects, unverified masking, score–content inconsistencies and the absence of patient validation. They do not establish general model superiority, response repeatability, clinical readiness or patient benefit.
Related
- Citation Fabrication and Accuracy in AI-Generated Psychiatric Research: A Comparative Evaluation of Four Contemporary Large Language Models
- Perceptions of artificial intelligence for radiographic image accuracy: a cross-sectional survey of radiology personnel
- Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
- A Multidimensional Evaluation of Large Language Model Responses to the 2024 ESC Hypertension Guidelines: A Comparative Study
- Performance of large language models in postoperative hip fracture rehabilitation counseling for older adults: a comparative evaluation of safety, accuracy, reliability, readability, and empathy
BibTeX
@article{kapoor2026expertrated,
title = {{Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation}},
author = {Deepika Kapoor and Deepanshu Garg and Ashutosh Nirola and M. K. Sunil and Pradhuman Verma and Santosh Kumar Tadakamadla},
journal = {Frontiers in Oral Health},
year = {2026},
month = oct,
doi = {10.3389/froh.2026.1937905},
url = {https://doi.org/10.3389/froh.2026.1937905},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Expert-rated accuracy and safety of patient-facing dental radiology responses from three AI platform configurations: a comparative evaluation
AU - Deepika Kapoor
AU - Deepanshu Garg
AU - Ashutosh Nirola
AU - M. K. Sunil
AU - Pradhuman Verma
AU - Santosh Kumar Tadakamadla
PY - 2026
DA - 2026/10/21
JO - Frontiers in Oral Health
DO - 10.3389/froh.2026.1937905
UR - https://doi.org/10.3389/froh.2026.1937905
AB - Background: Consumer-facing artificial intelligence platforms are increasingly used to obtain health information, but the quality of their responses to dental radiology questions requires evaluation. This study compared expert ratings of single responses generated by three selected consumer-facing AI platform configurations to common patient-facing dental radiology questions. Methods: Twenty expert-reviewed English-language questions were submitted using one standardized prompt to the investigator-recorded, user-visible ChatGPT-5.5 configuration with Thinking enabled, Gemini 3 Flash free configuration, and Microsoft Copilot 2026 free configuration during late April 2026. One response was collected from each platform for each question, producing 60 responses. Three Oral and Maxillofacial Radiologists independently rated coded responses with platform labels concealed, assessing accuracy, relevance, clarity, completeness, safety, triage appropriateness and misinformation. The 20 questions were treated as the primary paired experimental units. Evaluator-averaged scores were compared across platform configurations using Friedman tests followed by question-level exact paired comparisons, while individual evaluator ratings were used descriptively and for agreement assessment. Results: The selected ChatGPT configuration received more favourable ratings than Gemini and Copilot for accuracy, relevance, safety and misinformation. It also received more favourable clarity and completeness ratings than Copilot. Gemini received more favourable question-level ratings than Copilot for clarity and completeness, but not for relevance. No significant differences were observed between ChatGPT and Gemini for clarity or completeness. Triage was applicable only to Question 20 and was reported descriptively. Considerable ceiling effects and tied ratings were observed, including 100% maximum ratings for ChatGPT accuracy and safety. Conclusion: Under the specified testing conditions, expert ratings differed across the single outputs generated by the selected platform configurations. The findings are limited by unequal platform configurations, one generation per question, unavailable historical interface metadata, a relatively homogeneous informational question set, ceiling effects, unverified masking, score–content inconsistencies and the absence of patient validation. They do not establish general model superiority, response repeatability, clinical readiness or patient benefit.
SN - 2673-4842
ER -