AI-generated patient information leaflets for oral anticoagulants: quality, usability, readability, and reproducibility compared with FDA-referenced materials
5 affiliations
- 1School of Pharmaceutical Science, Lovely Professional University, Phagwara, Punjab 144411 India
- 2Department of Pharmacy, Hamad Medical Corporation, Doha, Qatar
- 3Regeneron Pharmaceuticals, Tarrytown, NY USA
- 4Department of Cardiology, Hamad Medical Corporation, Doha, Qatar
- 5Department of Pharmacology, Teerthanker Mahaveer College of Pharmacy, Moradabad, India
Abstract
Introduction: Oral anticoagulants are frequently associated with preventable hospital admissions, and patient education is essential to their safe and effective use. Patients increasingly use large language models (LLMs) for medication information, yet few studies have compared AI-generated outputs with regulatory materials or examined oral anticoagulants, a class in which misunderstanding can cause bleeding or thrombosis.
Aim: To assess the informational quality, usability, readability, and output reproducibility of AI-generated patient information leaflets (PILs) for oral anticoagulants compared with FDA-referenced patient materials.
Method: PILs for five oral anticoagulants (warfarin, apixaban, dabigatran, rivaroxaban, and edoxaban) were generated by ChatGPT, Gemini, and DeepSeek using a standardised zero-shot prompt based on FDA labelling templates; FDA-approved leaflets served as the comparator. Materials were anonymised and brand-blinded. Three clinical pharmacists independently evaluated each PIL using the Patient Education Materials Assessment Tool for Print Materials (PEMAT-P) and a modified DISCERN (mDISCERN), which assess informational quality, understandability, and actionability, but not pharmacological accuracy or clinical safety. Readability was assessed using seven validated indices. Output reproducibility was examined within the same day and at Day 1, 14, and 28.
Results: ChatGPT produced PILs of informational quality similar to FDA-referenced materials, both achieving a median mDISCERN score of (41/65); DeepSeek (34/65) and Gemini (33/65) scored significantly lower than ChatGPT; Gemini also scored significantly lower than the FDA-referenced materials. Understandability was acceptable for all sources, whereas actionability was limited, including in FDA materials, with no leaflet providing a patient summary or decision-support tool. All materials exceeded the recommended sixth- to eighth-grade range on most indices, with FDA leaflets the most complex. Output was stable across generations, with no significant within-model differences and the largest mean difference of 0.23 points.
Conclusion: Among the evaluated models, ChatGPT most closely matched FDA-referenced materials in the assessed informational domains and demonstrated stable output over 28 days. Shared deficiencies across AI-generated and FDA leaflets suggest broader limitations in written health information. This comparability was limited to the assessed informational domains and does not establish equivalence in pharmacological accuracy, clinical correctness, or patient safety. AI-generated PILs may serve as clinician-reviewed supplementary materials and should not be used as standalone tools without independent professional review and verification of clinical content.
Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1007/s11096-026-02203-2.
Related
- Quality, readability, and information fidelity of large language model–generated versus physician-authored informed-consent content for targeted therapies: a blinded, paired comparative study
- AI-generated high-intensity interval training information: a cross-sectional comparative evaluation of quality and readability
- Readability of Retina Patient Education Materials Generated With a Large Language Model
- Four large language models in parent-oriented education for pediatric flexible flatfoot: Quality, readability, hallucination, and reproducibility
- Nutrient Calculation Accuracy, Expert-Rated Clinical Appropriateness and Safety Indicators, and Reproducibility of Large Language Model-Generated Diet Plans for Polyendocrine Metabolic Ovarian Syndrome: A Real-Case-Based Evaluation
BibTeX
@article{khan2026aigenerated,
title = {{AI-generated patient information leaflets for oral anticoagulants: quality, usability, readability, and reproducibility compared with FDA-referenced materials}},
author = {Mohammed Amer Khan and Mohammed Maazuddin and Ammar Al Abdullah and Ihtisham Sultan and Hakam Abdallah Alzaeem and Mohammed Qintar and Krishana Kumar Sharma and Biplab Pal},
journal = {International Journal of Clinical Pharmacy},
year = {2026},
month = aug,
doi = {10.1007/s11096-026-02203-2},
url = {https://doi.org/10.1007/s11096-026-02203-2},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - JOUR
TI - AI-generated patient information leaflets for oral anticoagulants: quality, usability, readability, and reproducibility compared with FDA-referenced materials
AU - Mohammed Amer Khan
AU - Mohammed Maazuddin
AU - Ammar Al Abdullah
AU - Ihtisham Sultan
AU - Hakam Abdallah Alzaeem
AU - Mohammed Qintar
AU - Krishana Kumar Sharma
AU - Biplab Pal
PY - 2026
DA - 2026/08/06
JO - International Journal of Clinical Pharmacy
DO - 10.1007/s11096-026-02203-2
UR - https://doi.org/10.1007/s11096-026-02203-2
AB - Introduction: Oral anticoagulants are frequently associated with preventable hospital admissions, and patient education is essential to their safe and effective use. Patients increasingly use large language models (LLMs) for medication information, yet few studies have compared AI-generated outputs with regulatory materials or examined oral anticoagulants, a class in which misunderstanding can cause bleeding or thrombosis. Aim: To assess the informational quality, usability, readability, and output reproducibility of AI-generated patient information leaflets (PILs) for oral anticoagulants compared with FDA-referenced patient materials. Method: PILs for five oral anticoagulants (warfarin, apixaban, dabigatran, rivaroxaban, and edoxaban) were generated by ChatGPT, Gemini, and DeepSeek using a standardised zero-shot prompt based on FDA labelling templates; FDA-approved leaflets served as the comparator. Materials were anonymised and brand-blinded. Three clinical pharmacists independently evaluated each PIL using the Patient Education Materials Assessment Tool for Print Materials (PEMAT-P) and a modified DISCERN (mDISCERN), which assess informational quality, understandability, and actionability, but not pharmacological accuracy or clinical safety. Readability was assessed using seven validated indices. Output reproducibility was examined within the same day and at Day 1, 14, and 28. Results: ChatGPT produced PILs of informational quality similar to FDA-referenced materials, both achieving a median mDISCERN score of (41/65); DeepSeek (34/65) and Gemini (33/65) scored significantly lower than ChatGPT; Gemini also scored significantly lower than the FDA-referenced materials. Understandability was acceptable for all sources, whereas actionability was limited, including in FDA materials, with no leaflet providing a patient summary or decision-support tool. All materials exceeded the recommended sixth- to eighth-grade range on most indices, with FDA leaflets the most complex. Output was stable across generations, with no significant within-model differences and the largest mean difference of 0.23 points. Conclusion: Among the evaluated models, ChatGPT most closely matched FDA-referenced materials in the assessed informational domains and demonstrated stable output over 28 days. Shared deficiencies across AI-generated and FDA leaflets suggest broader limitations in written health information. This comparability was limited to the assessed informational domains and does not establish equivalence in pharmacological accuracy, clinical correctness, or patient safety. AI-generated PILs may serve as clinician-reviewed supplementary materials and should not be used as standalone tools without independent professional review and verification of clinical content. Supplementary Information: The online version contains supplementary material available at https://doi.org/10.1007/s11096-026-02203-2.
SN - 2210-7703
SN - 2210-7711
ER -