Quality, transparency, and readability of artificial intelligence chatbot responses to fibrous dysplasia-related searches: a cross-sectional comparative study
- 1Department of Orthopaedics, Yueyang Hospital Affiliated to Hunan Normal University
- 2Health Management Center, Yueyang Hospital Affiliated to Hunan Normal University
- 3Department of Pathology, Yueyang Hospital Affiliated to Hunan Normal University
Abstract
Introduction: Fibrous dysplasia (FD) ranges from incidental monostotic lesions to complex craniofacial, polyostotic, and syndromic disease. Short online searches may therefore represent different clinical questions. We compared the quality, transparency, and readability of FD-related responses from four consumer artificial intelligence (AI) chatbots.
Methods: In this cross-sectional online study, 10 unmodified English-language queries from a worldwide Google Trends dataset were submitted once to ChatGPT 5.5, Gemini 3.5 Flash, Microsoft Copilot, and Perplexity on August 11, 2026, yielding 40 responses. Responses were evaluated with DISCERN, a prespecified 20-item binary EQIP framework, GQS, and the JAMA benchmark criteria. Six readability indices were calculated. Between-model comparisons used Kruskal–Wallis tests with epsilon-squared effect sizes, followed where appropriate by Dunn post-hoc comparisons with Bonferroni adjustment.
Results: DISCERN, EQIP, JAMA, and GQS differed among chatbots ( p < 0.001, < 0.001, < 0.001, and 0.037, respectively). Perplexity had the highest median DISCERN [43.00 (41.00, 45.00)], EQIP [70.00 (66.25, 73.75)], and JAMA [1.00 (1.00, 1.00)] scores. All six readability indices differed among models (all p ≤ 0.037), and the formula-based scores indicated difficult text.
Discussion: The chatbots showed different profiles across the evaluated quality, transparency, and readability measures. Perplexity achieved higher scores on several evaluated instruments, while visible transparency cues remained limited across models. The findings describe a single-time-point, single-generation assessment of this selected query set.
Related
- Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
- Quality and readability of responses generated by popular large language models to frequently asked questions about menopause: A comparative study
- Quality and Readability of ChatGPT and Gemini Responses to Patient Questions About Systemic Isotretinoin: Comparative Evaluation Study.
- Patient expectations for artificial intelligence transparency and agency: A qualitative study and proportionate framework for clinical artificial intelligence implementation
- Comparison of ChatGPT, Gemini, and DeepSeek responses to questions on medication-related osteonecrosis of the jaw (MRONJ): a comparative analysis of expert-rated clinical quality, readability, response time, and clinical safety risk.
BibTeX
@article{ying2026quality,
title = {{Quality, transparency, and readability of artificial intelligence chatbot responses to fibrous dysplasia-related searches: a cross-sectional comparative study}},
author = {Dawen Ying and Jinxi Hu and Qifang Zeng and Minzheng Ying and Min Yi and Zhizheng Xiong},
journal = {Frontiers in Public Health},
year = {2026},
month = nov,
doi = {10.3389/fpubh.2026.1992509},
url = {https://www.frontiersin.org/articles/10.3389/fpubh.2026.1992509/full},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Quality, transparency, and readability of artificial intelligence chatbot responses to fibrous dysplasia-related searches: a cross-sectional comparative study
AU - Dawen Ying
AU - Jinxi Hu
AU - Qifang Zeng
AU - Minzheng Ying
AU - Min Yi
AU - Zhizheng Xiong
PY - 2026
DA - 2026/11/04
JO - Frontiers in Public Health
DO - 10.3389/fpubh.2026.1992509
UR - https://www.frontiersin.org/articles/10.3389/fpubh.2026.1992509/full
AB - Introduction: Fibrous dysplasia (FD) ranges from incidental monostotic lesions to complex craniofacial, polyostotic, and syndromic disease. Short online searches may therefore represent different clinical questions. We compared the quality, transparency, and readability of FD-related responses from four consumer artificial intelligence (AI) chatbots. Methods: In this cross-sectional online study, 10 unmodified English-language queries from a worldwide Google Trends dataset were submitted once to ChatGPT 5.5, Gemini 3.5 Flash, Microsoft Copilot, and Perplexity on August 11, 2026, yielding 40 responses. Responses were evaluated with DISCERN, a prespecified 20-item binary EQIP framework, GQS, and the JAMA benchmark criteria. Six readability indices were calculated. Between-model comparisons used Kruskal–Wallis tests with epsilon-squared effect sizes, followed where appropriate by Dunn post-hoc comparisons with Bonferroni adjustment. Results: DISCERN, EQIP, JAMA, and GQS differed among chatbots ( p < 0.001, < 0.001, < 0.001, and 0.037, respectively). Perplexity had the highest median DISCERN [43.00 (41.00, 45.00)], EQIP [70.00 (66.25, 73.75)], and JAMA [1.00 (1.00, 1.00)] scores. All six readability indices differed among models (all p ≤ 0.037), and the formula-based scores indicated difficult text. Discussion: The chatbots showed different profiles across the evaluated quality, transparency, and readability measures. Perplexity achieved higher scores on several evaluated instruments, while visible transparency cues remained limited across models. The findings describe a single-time-point, single-generation assessment of this selected query set.
SN - 2296-2565
ER -