Comparative Analysis of Large Language Model Outputs for Pre-Hypnotherapy Support in Irritable Bowel Syndrome: An Experimental Feasibility Study
- 1Faculty of Medicine, MAHSA University, Bandar Saujana Putra Campus, Kuala Langat Selangor Malaysia
Abstract
Background/Aims: Gut-directed hypnotherapy can assist with irritable bowel syndrome, but access remains limited. This feasibility study conducted an exploratory comparison of four large language models (LLMs) to evaluate the readability and information quality of model generated responses to common patient questions about pre- hypnotherapy support.
Methods: Four LLMs (ChatGPT-4o, Claude 3.7 Sonnet, Gemini, and DeepSeek-R1) were queried using a pre-specified set of 14 standardized patient-oriented prompts. Responses were assessed for readability using validated indices (FKGL, FRES, and CLI) and for information quality using the DISCERN instrument and five item Likert scale, rated by a single evaluator. Paired statistical tests were used, and results are reported with effect sizes and confidence intervals.
Results: In this exploratory evaluation, DeepSeek-R1 and ChatGPT-4o generated responses with relatively higher scores across readability metrics (Flesch-Kincaid Grade Level, Flesch Reading Ease Score, and Coleman-Liau Index) and information quality measures (DISCERN and Likert), which are distinct constructs assessed by separate instruments. Claude 3.7 Sonnet produced denser with lower readability and reliability scores, and Gemini showed intermediate performance. Interpretation is limited using automated readability metrics and single-rater design. As the DISCERN and Likert scores were generated by a single rater, statistical comparisons involving these measures cannot be considered definitive. The results below are presented solely to illustrate observable trends within this exploratory dataset.
Conclusions: LLMs show potential for generating patient oriented pre-hypnotherapy information, but this feasibility study is not sufficient to determine clinical applicability. The findings primarily highlight the need for expanded evaluation with multiple expert raters, patient participants, and more robust quality assessment methods before considering any practical or clinical use.
Related
- Test-Retest Reliability of Large Language Models and Their Concordance With Human Experts in Rubric-Based Scoring of Clinical Summary Statements: Experimental Comparative Study.
- Minithoracotomy approach for redo mitral valve surgery: Feasibility, operative outcomes, and comparative analysis of myocardial protection techniques
- Feasibility of Privacy-preserving Locally Hosted Language Models for Assessing Medical Student Reflective Writing.
- Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency—A Cross-Sectional Comparative Benchmark Study
- Performance of large language models in prescription review: a comparative study
BibTeX
@article{godavarthy2026comparative,
title = {{Comparative Analysis of Large Language Model Outputs for Pre-Hypnotherapy Support in Irritable Bowel Syndrome: An Experimental Feasibility Study}},
author = {Praghya Godavarthy and Manish Ravi Iyer and Ravinraj Munnavan and Tanuri Levanya Halhaturage and Gauri Krishnaswamy and Ramisa Fariha},
journal = {Journal of Neurogastroenterology and Motility},
year = {2026},
month = oct,
doi = {10.5056/jnm25145},
url = {https://doi.org/10.5056/jnm25145},
note = {CC BY-NC 4.0},
}
RIS
TY - JOUR
TI - Comparative Analysis of Large Language Model Outputs for Pre-Hypnotherapy Support in Irritable Bowel Syndrome: An Experimental Feasibility Study
AU - Praghya Godavarthy
AU - Manish Ravi Iyer
AU - Ravinraj Munnavan
AU - Tanuri Levanya Halhaturage
AU - Gauri Krishnaswamy
AU - Ramisa Fariha
PY - 2026
DA - 2026/10/31
JO - Journal of Neurogastroenterology and Motility
DO - 10.5056/jnm25145
UR - https://doi.org/10.5056/jnm25145
AB - Background/Aims: Gut-directed hypnotherapy can assist with irritable bowel syndrome, but access remains limited. This feasibility study conducted an exploratory comparison of four large language models (LLMs) to evaluate the readability and information quality of model generated responses to common patient questions about pre- hypnotherapy support. Methods: Four LLMs (ChatGPT-4o, Claude 3.7 Sonnet, Gemini, and DeepSeek-R1) were queried using a pre-specified set of 14 standardized patient-oriented prompts. Responses were assessed for readability using validated indices (FKGL, FRES, and CLI) and for information quality using the DISCERN instrument and five item Likert scale, rated by a single evaluator. Paired statistical tests were used, and results are reported with effect sizes and confidence intervals. Results: In this exploratory evaluation, DeepSeek-R1 and ChatGPT-4o generated responses with relatively higher scores across readability metrics (Flesch-Kincaid Grade Level, Flesch Reading Ease Score, and Coleman-Liau Index) and information quality measures (DISCERN and Likert), which are distinct constructs assessed by separate instruments. Claude 3.7 Sonnet produced denser with lower readability and reliability scores, and Gemini showed intermediate performance. Interpretation is limited using automated readability metrics and single-rater design. As the DISCERN and Likert scores were generated by a single rater, statistical comparisons involving these measures cannot be considered definitive. The results below are presented solely to illustrate observable trends within this exploratory dataset. Conclusions: LLMs show potential for generating patient oriented pre-hypnotherapy information, but this feasibility study is not sufficient to determine clinical applicability. The findings primarily highlight the need for expanded evaluation with multiple expert raters, patient participants, and more robust quality assessment methods before considering any practical or clinical use.
SN - 2093-0879
SN - 2093-0887
ER -