Guideline adherence and expert-rated potential safety concerns in AI-generated dietary advice for patients with cirrhosis: a comparative evaluation of large language models.
- 1Huadu District People's Hospital of Guangzhou, Guangzhou, Guangdong, China.
Abstract
Background
Patients with cirrhosis require complication-specific nutritional counseling. Large language models are increasingly used for health information seeking, but the guideline adherence and potential safety concerns of their dietary advice for cirrhosis remain uncertain.Methods
Thirty-six standardized simulated cases represented compensated cirrhosis, cirrhosis with ascites, and hepatic encephalopathy recovery. Each case was submitted once to DeepSeek, ChatGPT, Qwen, and Claude through their official web interfaces, generating 144 English-language recommendations. Three reviewers performed de novo model-blinded assessments using a locked framework. The primary outcome was response-level Guideline Adherence; secondary outcomes included Error Count, Any Major Error, and Harm Flag.Results
Mean Guideline Adherence was 7.00 ± 0.00 for ChatGPT, 6.97 ± 0.17 for Claude, 6.94 ± 0.19 for DeepSeek, and 6.44 ± 0.84 for Qwen (Friedman p < 0.001). In hepatic encephalopathy recovery, Qwen scored 5.36 ± 0.54 and was lower than each other model after Holm adjustment (all adjusted p = 0.009). Any Major Error occurred in 0 ChatGPT, 2 Claude, 4 DeepSeek, and 11 Qwen responses; corresponding Harm Flag counts were 0, 1, 2, and 8. Most events occurred during hepatic encephalopathy recovery and involved protein-related recommendations.Conclusion
Under standardized English-language structured prompting, all four models generally reproduced core cirrhosis nutrition principles. However, Qwen showed lower adherence and more frequent protein-related errors in hepatic encephalopathy recovery. Expert verification remains necessary before patient-facing or individualized use.
Related
- Citation Fabrication and Accuracy in AI-Generated Psychiatric Research: A Comparative Evaluation of Four Contemporary Large Language Models
- A multi-dimensional comparative evaluation of ten injectable neuroprotective agents for acute ischemic stroke based on the Chinese guideline for drug selection
- Comparative evaluation of large language models for guideline-concordant decision support in colorectal cancer multidisciplinary team care: A source-masked retrospective study
- Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study.
- Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency—A Cross-Sectional Comparative Benchmark Study
BibTeX
@article{liu2026guideline,
title = {{Guideline adherence and expert-rated potential safety concerns in AI-generated dietary advice for patients with cirrhosis: a comparative evaluation of large language models.}},
author = {Qiong Liu and Guojun Liang and Xiaofang Liu and Yang Song and Hongsheng Wu},
journal = {Frontiers in nutrition},
year = {2026},
month = oct,
doi = {10.3389/fnut.2026.1893597},
url = {https://europepmc.org/article/MED/42924246},
note = {CC BY},
}
RIS
TY - JOUR
TI - Guideline adherence and expert-rated potential safety concerns in AI-generated dietary advice for patients with cirrhosis: a comparative evaluation of large language models.
AU - Qiong Liu
AU - Guojun Liang
AU - Xiaofang Liu
AU - Yang Song
AU - Hongsheng Wu
PY - 2026
DA - 2026/10/08
JO - Frontiers in nutrition
DO - 10.3389/fnut.2026.1893597
UR - https://europepmc.org/article/MED/42924246
AB - Background Patients with cirrhosis require complication-specific nutritional counseling. Large language models are increasingly used for health information seeking, but the guideline adherence and potential safety concerns of their dietary advice for cirrhosis remain uncertain.Methods Thirty-six standardized simulated cases represented compensated cirrhosis, cirrhosis with ascites, and hepatic encephalopathy recovery. Each case was submitted once to DeepSeek, ChatGPT, Qwen, and Claude through their official web interfaces, generating 144 English-language recommendations. Three reviewers performed de novo model-blinded assessments using a locked framework. The primary outcome was response-level Guideline Adherence; secondary outcomes included Error Count, Any Major Error, and Harm Flag.Results Mean Guideline Adherence was 7.00 ± 0.00 for ChatGPT, 6.97 ± 0.17 for Claude, 6.94 ± 0.19 for DeepSeek, and 6.44 ± 0.84 for Qwen (Friedman p < 0.001). In hepatic encephalopathy recovery, Qwen scored 5.36 ± 0.54 and was lower than each other model after Holm adjustment (all adjusted p = 0.009). Any Major Error occurred in 0 ChatGPT, 2 Claude, 4 DeepSeek, and 11 Qwen responses; corresponding Harm Flag counts were 0, 1, 2, and 8. Most events occurred during hepatic encephalopathy recovery and involved protein-related recommendations.Conclusion Under standardized English-language structured prompting, all four models generally reproduced core cirrhosis nutrition principles. However, Qwen showed lower adherence and more frequent protein-related errors in hepatic encephalopathy recovery. Expert verification remains necessary before patient-facing or individualized use.
SN - 2296-861X
ER -