Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated 18 min ago
Peer-reviewedCC BY 4.0Frontiers in Public Health

Evaluating AI chatbots as sources of public-facing encephalitis health information: a cross-sectional comparative study of quality, transparency, and readability

  • Wen Ye1
  • Yu Wang2
  • Hongli Liu1
  • Xiaoge Zhang1
  • Tingting Liu1
  • Wenhao Shi3
  1. 1Department of Pediatric Neurology, Northwest Women’s and Children’s Hospital
  2. 2Department of Pharmacy, Northwest University First Hospital
  3. 3Assisted Reproduction Center, Northwest Women’s and Children’s Hospital
Open at sourceDOICite

Abstract

Background: Publicly accessible AI chatbots are increasingly used as interactive sources of health information, yet their suitability for patient-facing communication depends on information quality, source transparency, and readability. Encephalitis is a clinically heterogeneous and potentially serious condition for which patients and caregivers may seek information about symptoms, causes, diagnosis, imaging, and treatment.

Objective: To evaluate the quality, transparency, and readability of encephalitis health information generated by four publicly accessible AI chatbots and to consider the implications for patient-oriented digital health communication.

Methods: This web-based cross-sectional study evaluated 32 core encephalitis-related queries identified using Google Trends, each submitted once to four LLM chatbots: ChatGPT, Perplexity, Gemini, and Copilot. The responses were assessed using DISCERN, Ensuring Quality Information for Patients (EQIP), the JAMA benchmark criteria, and the Global Quality Scale (GQS), along with six validated readability indices. Differences among chatbots were analyzed using the Friedman test with paired post hoc comparisons and Holm correction.

Results: All 32 queries were analyzed. DISCERN, EQIP, and JAMA scores differed significantly among the four chatbots (all p < 0.001), whereas GQS scores did not ( p = 0.090). Perplexity and Copilot had higher median DISCERN and EQIP scores than ChatGPT and Gemini, but pairwise differences were not uniform across all measures. JAMA scores were higher for Perplexity and Copilot than for ChatGPT and Gemini in most pairwise comparisons. None of the four chatbots met the prespecified sixth-grade readability benchmarks.

Conclusion: In this single-time-point evaluation of 32 English-language encephalitis queries, chatbots differed across selected information-quality and transparency measures, while none met the prespecified sixth-grade readability benchmark. Further evaluation of response stability, factual accuracy, clinical safety, and patient-centered outcomes is needed to establish the implications for patient-oriented use.

Related

  1. Quality, transparency, and readability of artificial intelligence chatbot responses to fibrous dysplasia-related searches: a cross-sectional comparative study
    Dawen Ying, Jinxi Hu, Qifang Zeng +3
    Frontiers in Public HealthPeer-reviewedCC BY 4.0Epidemiology & public healthCrossref
  2. Generative AI chatbots as sources of phimosis-related consumer health information: a cross-sectional comparison of quality, source transparency, and readability.
    Qinghua Luo, Leihua Cao, Pan Ding
    Frontiers in public healthPeer-reviewedCC BYFull textEpidemiology & public healthEurope PMCCrossref
  3. Quality, Readability, and Content Deficiencies of Three Large Language Models in Bee Sting Health Information: A Comparative Evaluation with Patient Perspectives
    R. Xu, L. Yang, X. Zhang +3
    medRxivPreprintCC BYFull textEpidemiology & public healthmedRxiv
  4. Evaluating the reliability, quality, and readability of AI-generated patient education on hallux valgus: a comparative study of large language models
    Ali Can Koluman, Ebru Aloglu Ciftci, Mehmet Utku Ciftci +2
    BMC Medical Informatics and Decision MakingPeer-reviewedCC BY 4.0Full text1 citedEpidemiology & public healthPMCDOAJ
  5. Quality, readability, and information fidelity of large language model–generated versus physician-authored informed-consent content for targeted therapies: a blinded, paired comparative study
    Yue Xie, Yuan Gao, Jiajing Peng +6
    Frontiers in PharmacologyPeer-reviewedCC BY 4.0Pharmacology & toxicologyFrontiersCrossref

Cite

BibTeXRIS
BibTeX
@article{ye2026evaluating,
  title = {{Evaluating AI chatbots as sources of public-facing encephalitis health information: a cross-sectional comparative study of quality, transparency, and readability}},
  author = {Wen Ye and Yu Wang and Hongli Liu and Xiaoge Zhang and Tingting Liu and Wenhao Shi},
  journal = {Frontiers in Public Health},
  year = {2026},
  month = nov,
  doi = {10.3389/fpubh.2026.1986483},
  url = {https://www.frontiersin.org/articles/10.3389/fpubh.2026.1986483/full},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Evaluating AI chatbots as sources of public-facing encephalitis health information: a cross-sectional comparative study of quality, transparency, and readability
AU  - Wen Ye
AU  - Yu Wang
AU  - Hongli Liu
AU  - Xiaoge Zhang
AU  - Tingting Liu
AU  - Wenhao Shi
PY  - 2026
DA  - 2026/11/09
JO  - Frontiers in Public Health
DO  - 10.3389/fpubh.2026.1986483
UR  - https://www.frontiersin.org/articles/10.3389/fpubh.2026.1986483/full
AB  - Background: Publicly accessible AI chatbots are increasingly used as interactive sources of health information, yet their suitability for patient-facing communication depends on information quality, source transparency, and readability. Encephalitis is a clinically heterogeneous and potentially serious condition for which patients and caregivers may seek information about symptoms, causes, diagnosis, imaging, and treatment.  Objective: To evaluate the quality, transparency, and readability of encephalitis health information generated by four publicly accessible AI chatbots and to consider the implications for patient-oriented digital health communication.  Methods: This web-based cross-sectional study evaluated 32 core encephalitis-related queries identified using Google Trends, each submitted once to four LLM chatbots: ChatGPT, Perplexity, Gemini, and Copilot. The responses were assessed using DISCERN, Ensuring Quality Information for Patients (EQIP), the JAMA benchmark criteria, and the Global Quality Scale (GQS), along with six validated readability indices. Differences among chatbots were analyzed using the Friedman test with paired post hoc comparisons and Holm correction.  Results: All 32 queries were analyzed. DISCERN, EQIP, and JAMA scores differed significantly among the four chatbots (all p < 0.001), whereas GQS scores did not ( p = 0.090). Perplexity and Copilot had higher median DISCERN and EQIP scores than ChatGPT and Gemini, but pairwise differences were not uniform across all measures. JAMA scores were higher for Perplexity and Copilot than for ChatGPT and Gemini in most pairwise comparisons. None of the four chatbots met the prespecified sixth-grade readability benchmarks.  Conclusion: In this single-time-point evaluation of 32 English-language encephalitis queries, chatbots differed across selected information-quality and transparency measures, while none met the prespecified sixth-grade readability benchmark. Further evaluation of response stability, factual accuracy, clinical safety, and patient-centered outcomes is needed to establish the implications for patient-oriented use.
SN  - 2296-2565
ER  -