Prevalent knowledge omissions: An unseen risk of LLMs in patient-friendly adaptation of medical literature
- 1School of Information Engineering, Minzu University of China, Beijing, China
- 2Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE, Minzu University of China, Beijing, China
Abstract
Large language models (LLMs) can produce patient-friendly adaptations, such as simplifying specialized medical literature for readers with a middle school education. However, any distortion of knowledge during this process may have serious consequences. Current evaluations overlook such risks. In this study, we proposes a knowledge triplet comparison (KTC) method to assess knowledge distortion by systematically quantifying the retention and alteration of medical knowledge before and after text adaptation. Using the Chemical-Disease Relations dataset, this study evaluates the performance of six LLMs across eight adaptation scenarios (e.g., adjusting reading level, modifying text length) and draws the following key conclusions: First, from a quantitative perspective, knowledge distortion is prevalent in all adaptation scenarios, with the models retaining only approximately half of the original medical knowledge. Additionally, about 10% more knowledge is added. Second, the distribution of knowledge distortion follows a clear categorical distribution pattern, with etiology and pathological mechanisms, as well as complications and risk factors, being the most frequently distortion types. Third, targeted corrections through prompt engineering have no significant impact on the quantity and categorical distribution patterns of knowledge distortion, indicating that knowledge distortion is persistent and regular. This study shows that, in addition to the more obvious risk of hallucinations caused by a small amount of added knowledge, patient-friendly rewrites by LLMs also pose a significant risk of subtle yet potentially serious knowledge omissions. Furthermore, this research reveals the regularity of knowledge distortion, laying the groundwork for further investigation into the patterns of knowledge distortion and the development of solutions to this problem. Author summary: Many patients look to medical research articles to better understand their conditions and treatment options. Recently, large language models (LLMs) have been used to rewrite complex medical texts into more accessible language. While this approach may improve understanding, it also raises concerns about whether important medical knowledge is preserved. In this study, we introduce a method called Knowledge Triplet Comparison (KTC) to measure how medical knowledge changes when texts are rewritten for patients. Using data on relationships between chemicals and diseases, we evaluated six widely used LLMs across several common rewriting tasks, such as simplifying language or shortening text. We found that important medical knowledge was frequently altered or lost. On average, only about half of the original knowledge was retained, while some previously unstated knowledge was occasionally introduced. These changes most often affected explanations of disease causes, underlying mechanisms, complications, and risk factors. Notably, even prompts designed to encourage accuracy did not substantially reduce these issues. Our findings highlight a potential risk in using AI to simplify medical knowledge: key details may be silently omitted or changed, which could lead to misunderstanding. Care should therefore be taken when using these tools to communicate medical knowledge to patients.
Related
- Traditional Ecological Relationship Family sharing an abundant home within nature requires recurring mutual adaptation rather than the extraction of knowledge by passing visitors
- Place-based climate adaptation: How Atlantic Canadian communities translate local knowledge into action
- A monoallelic immunopeptidome atlas of 10 prevalent canine DLA-88 alleles
- Prevalent α-chain–centric TCR metaclones shape specificity and focus early SARS-CoV-2 T cell responses
- Highly prevalent cargo genes of F plasmids harbored in pathogenic Escherichia coli
BibTeX
@article{chen2026prevalent,
title = {{Prevalent knowledge omissions: An unseen risk of LLMs in patient-friendly adaptation of medical literature}},
author = {Mei Chen and Shibin Wang},
journal = {PLOS Digital Health},
year = {2026},
month = oct,
doi = {10.1371/journal.pdig.0001739},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001739},
note = {CC BY},
}
RIS
TY - JOUR
TI - Prevalent knowledge omissions: An unseen risk of LLMs in patient-friendly adaptation of medical literature
AU - Mei Chen
AU - Shibin Wang
PY - 2026
DA - 2026/10/01
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001739
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001739
AB - Large language models (LLMs) can produce patient-friendly adaptations, such as simplifying specialized medical literature for readers with a middle school education. However, any distortion of knowledge during this process may have serious consequences. Current evaluations overlook such risks. In this study, we proposes a knowledge triplet comparison (KTC) method to assess knowledge distortion by systematically quantifying the retention and alteration of medical knowledge before and after text adaptation. Using the Chemical-Disease Relations dataset, this study evaluates the performance of six LLMs across eight adaptation scenarios (e.g., adjusting reading level, modifying text length) and draws the following key conclusions: First, from a quantitative perspective, knowledge distortion is prevalent in all adaptation scenarios, with the models retaining only approximately half of the original medical knowledge. Additionally, about 10% more knowledge is added. Second, the distribution of knowledge distortion follows a clear categorical distribution pattern, with etiology and pathological mechanisms, as well as complications and risk factors, being the most frequently distortion types. Third, targeted corrections through prompt engineering have no significant impact on the quantity and categorical distribution patterns of knowledge distortion, indicating that knowledge distortion is persistent and regular. This study shows that, in addition to the more obvious risk of hallucinations caused by a small amount of added knowledge, patient-friendly rewrites by LLMs also pose a significant risk of subtle yet potentially serious knowledge omissions. Furthermore, this research reveals the regularity of knowledge distortion, laying the groundwork for further investigation into the patterns of knowledge distortion and the development of solutions to this problem. Author summary: Many patients look to medical research articles to better understand their conditions and treatment options. Recently, large language models (LLMs) have been used to rewrite complex medical texts into more accessible language. While this approach may improve understanding, it also raises concerns about whether important medical knowledge is preserved. In this study, we introduce a method called Knowledge Triplet Comparison (KTC) to measure how medical knowledge changes when texts are rewritten for patients. Using data on relationships between chemicals and diseases, we evaluated six widely used LLMs across several common rewriting tasks, such as simplifying language or shortening text. We found that important medical knowledge was frequently altered or lost. On average, only about half of the original knowledge was retained, while some previously unstated knowledge was occasionally introduced. These changes most often affected explanations of disease causes, underlying mechanisms, complications, and risk factors. Notably, even prompts designed to encourage accuracy did not substantially reduce these issues. Our findings highlight a potential risk in using AI to simplify medical knowledge: key details may be silently omitted or changed, which could lead to misunderstanding. Care should therefore be taken when using these tools to communicate medical knowledge to patients.
SN - 2767-3170
ER -