Patient-facing large language model assistants for chronic constipation education: A comparative study of expert evaluation, readability, and patient acceptability
Abstract
Objective To compare four patient-facing AI assistants for chronic constipation education across expert-rated quality, objective readability, and patient-reported acceptability. Methods In this two-phase study, 15 common questions and two clinical scenarios were submitted once to ChatGPT-4o, DeepSeek V3, Qwen3, and MedGo V1 using the same standardized Chinese prompt. Six blinded experts rated 68 responses across six domains. Objective readability was assessed using Chinese character count, reading difficulty score, and recommended reading age. In Phase 2, 30 patients each posed three questions; the investigator entered each question into all four assistants in a fixed sequence, and patients rated eight acceptability domains. Friedman tests were used; significant expert and patient results underwent Holm-adjusted Wilcoxon signed-rank tests. Results Accuracy ( p = .175) and safety ( p = .068) did not differ significantly across models. Domain specificity, expert-rated readability, humanity, and actionability differed significantly. Qwen3 scored higher than the other models for domain specificity and actionability in adjusted comparisons. MedGo V1 produced the shortest responses and had the lowest mean reading difficulty and recommended reading age. Patient ratings showed a different pattern: DeepSeek V3 had the highest mean overall acceptability (4.22 ± 0.19), exceeding Qwen3 (adjusted p < .001) and MedGo V1 (adjusted p = .044), but not ChatGPT-4o (adjusted p = .053). Conclusion Expert-rated quality, text complexity, and short-term patient acceptability were not fully aligned. No single assistant consistently outperformed the others. These findings describe model-specific performance profiles but do not establish clinical effectiveness, implementation readiness, or superiority of general-purpose versus medical-specialized systems.
Related
- Quality, Readability, Understandability, and Actionability of AI Chatbots’ Responses to Dental Patient Questions: A Comparative Evaluation
- Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency—A Cross-Sectional Comparative Benchmark Study
- Quality, Readability, and Content Deficiencies of Three Large Language Models in Bee Sting Health Information: A Comparative Evaluation with Patient Perspectives
- Comparative evaluation of large language models for esophageal cancer patient education: quality, reliability, readability, understandability, and actionability.
- Quality and Readability of ChatGPT and Gemini Responses to Patient Questions About Systemic Isotretinoin: Comparative Evaluation Study.
BibTeX
@article{msc2026patientfacing,
title = {{Patient-facing large language model assistants for chronic constipation education: A comparative study of expert evaluation, readability, and patient acceptability}},
author = {Yanfen Gu MSc and Shiqi Qiang MSc and Zehui Xu PhD and Haitao Zhang MSc and Xiaoxiao Peng BS and Haiping Yu PhD and Meidong Xu PhD},
journal = {Digital Health},
year = {2026},
month = oct,
doi = {10.1177/20552076261497162},
url = {https://doi.org/10.1177/20552076261497162},
note = {CC BY},
}
RIS
TY - JOUR
TI - Patient-facing large language model assistants for chronic constipation education: A comparative study of expert evaluation, readability, and patient acceptability
AU - Yanfen Gu MSc
AU - Shiqi Qiang MSc
AU - Zehui Xu PhD
AU - Haitao Zhang MSc
AU - Xiaoxiao Peng BS
AU - Haiping Yu PhD
AU - Meidong Xu PhD
PY - 2026
DA - 2026/10/24
JO - Digital Health
DO - 10.1177/20552076261497162
UR - https://doi.org/10.1177/20552076261497162
AB - Objective To compare four patient-facing AI assistants for chronic constipation education across expert-rated quality, objective readability, and patient-reported acceptability. Methods In this two-phase study, 15 common questions and two clinical scenarios were submitted once to ChatGPT-4o, DeepSeek V3, Qwen3, and MedGo V1 using the same standardized Chinese prompt. Six blinded experts rated 68 responses across six domains. Objective readability was assessed using Chinese character count, reading difficulty score, and recommended reading age. In Phase 2, 30 patients each posed three questions; the investigator entered each question into all four assistants in a fixed sequence, and patients rated eight acceptability domains. Friedman tests were used; significant expert and patient results underwent Holm-adjusted Wilcoxon signed-rank tests. Results Accuracy ( p = .175) and safety ( p = .068) did not differ significantly across models. Domain specificity, expert-rated readability, humanity, and actionability differed significantly. Qwen3 scored higher than the other models for domain specificity and actionability in adjusted comparisons. MedGo V1 produced the shortest responses and had the lowest mean reading difficulty and recommended reading age. Patient ratings showed a different pattern: DeepSeek V3 had the highest mean overall acceptability (4.22 ± 0.19), exceeding Qwen3 (adjusted p < .001) and MedGo V1 (adjusted p = .044), but not ChatGPT-4o (adjusted p = .053). Conclusion Expert-rated quality, text complexity, and short-term patient acceptability were not fully aligned. No single assistant consistently outperformed the others. These findings describe model-specific performance profiles but do not establish clinical effectiveness, implementation readiness, or superiority of general-purpose versus medical-specialized systems.
SN - 2055-2076
ER -