{ "abstract": "Radiology reports are often filled with medical jargon that limits patient understanding. Lay summaries can improve understanding but are time-consuming for healthcare providers to create. The objective of this study is to explore the use of tailored prompts for five Large Language Models (LLMs) in generating lay summaries from radiology reports. Using 100 reports from the publicly available “BioNLP 2023 report summarization” dataset, lay summaries were generated by each LLM, under select prompting styles [Few-Shot (GPT-4), Generated Knowledge (GPT-4o mini, Gemini 1.5 – Pro, Gemini 1.5 – Flash), and Zero-Shot (Llama 3.1)] informed by a pilot work. The summaries were evaluated using a mixed-method framework: subjective assessment (Likert statements) by blinded experts (n = 2 radiology fellows) and Large Reasoning Models (LRMs) [(Gemini 2.5 – Pro (LRM 1); GPT-oss-120b (LRM 2)], and readability metrics (Flesch-Kincaid Grade Level and Flesch Reading Ease). Using percentage agreement of Likert statements, the LLM-prompt combinations’ performances were ranked, and Friedman and post-hoc Nemenyi tests were conducted. Gemini 1.5 - Flash and - Pro (generated knowledge) were rated highest by human experts and LRMs for generating actionable lay summaries that require minimal supervision [P < 4.97 × 10-2 (Rater 1); P < 9.03 × 10-21 (Rater 2), P < 6.90 × 10-15 (LRM 1), P < 2.760 × 10-5 (LRM 2). GPT-4 (few-shot) achieved the highest human-rated accuracy (98%), while Gemini 1.5 – Flash (LRM 1-rated: 95%) and Gemini 1.5 – Pro (LRM 2-rated: 91%) ranked first in LRM-rated accuracy. Gemini 1.5 - Pro produced the most accessible summaries (Flesch-Kincaid Grade Level: 7.55 ± 1.38, Flesch Reading Ease: 67.84 ± 7.78). Strong agreement was observed between experts and LRMs [0.96% (LRM 1) and 3.4% (LRM 2) complete disagreement]. Overall, this study highlights Gemini-models with generated knowledge prompts and the potential of LRM evaluators in assessing LLM-generated lay summaries. Author summary: Creating plain-language medical summaries for patients is important, but it adds extra work for healthcare providers. While previously Artificial Intelligence systems, Large Language Models (LLM), were used to create patient-friendly summaries, this study is the first to explore how tailored prompt styles affect the model performance in generating summaries from radiology reports. Five models were prompted using pilot study-informed tailored styles: 1. What the model “knows” (Generated knowledge prompt for Gemini 1.5 - Flash, Gemini 1.5 – Pro and GPT-4o mini), 2. Guided examples (Few-shot for GPT-4) and 3. Only instructions (Zero-shot for Llama 3.1), to generate summaries for patients. These summaries were evaluated by human experts (radiology fellows), Gemini 2.5 – Pro and GPT-oss-120b [Large Reasoning Models (LRM)] and reading metrics. The results show that Gemini 1.5 – Flash and Gemini 1.5 – Pro with Generated knowledge prompting produced 7th-9th grade level summaries that had actionable and beneficial information and were more trusted by experts. We found that GPT-4 with Few-shot prompting produced the most accurate lay summaries according to the human raters. There was a strong similarity in ratings between the human experts and LRM evaluators. These findings show the potential of using AI to generate and evaluate clinical text summarization.", "abstract_shown": true, "affiliations": [ "MSc eHealth Program, McMaster University, Hamilton, Ontario, Canada", "Department of Medical Imaging, McMaster University, Hamilton, Ontario, Canada", "Department of Diagnostic Imaging, Juravinski Hospital and Cancer Centre, Hamilton Health Sciences, Hamilton, Ontario, Canada", "Escarpment Cancer Research Institute, Hamilton Health Sciences and McMaster University, Hamilton, Ontario, Canada", "Health Information Research Unit, Department of Health Research Methods, Evidence, and Impact, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada", "Department of Oncology, McMaster University, Juravinski Cancer Centre, Hamilton, Ontario, Canada", "CentRE for dAta science and digiTal hEalth (CREATE), Hamilton Health Sciences, Hamilton, Ontario, Canada" ], "article_type": "Research Article", "arxiv": null, "author_total": 7, "authors": [ { "affs": [ 0 ], "name": "Nanziba Tasneem" }, { "affs": [ 1 ], "name": "Christian B. van der Pol" }, { "affs": [ 2 ], "name": "Ambreen Zahoor" }, { "affs": [ 3 ], "name": "Nitin Juggath" }, { "affs": [ 4 ], "name": "Kyle McGowan" }, { "affs": [ 5 ], "name": "Cynthia Lokker" }, { "affs": [ 6 ], "name": "Ashirbani Saha" } ], "bibtex": "https://research.draconitic.com/paper/2bb88xbb0mbwt.bib", "cited_by": 0, "doi": "10.1371/journal.pdig.0001672", "fulltext_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001672", "id": "2bb88xbb0mbwt", "issns": [ "2767-3170" ], "landing_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001672", "language": null, "licence": "CC-BY-4.0", "licence_label": "CC BY 4.0", "licence_source": "journal", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "openalex", "2718|Artificial Intelligence in Healthcare and Education" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Language" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Language" ], [ "plos", "/Computer and information sciences/Artificial intelligence" ], [ "plos", "/Medicine and health sciences/Diagnostic medicine/Diagnostic radiology/Bone imaging/X-ray radiography" ], [ "plos", "/Medicine and health sciences/Diagnostic medicine/Diagnostic radiology/X-ray radiography" ], [ "plos", "/Medicine and health sciences/Health care/Patient advocacy" ], [ "plos", "/Medicine and health sciences/Radiology and imaging" ], [ "plos", "/Medicine and health sciences/Radiology and imaging/Diagnostic radiology/Bone imaging/X-ray radiography" ], [ "plos", "/Medicine and health sciences/Radiology and imaging/Diagnostic radiology/X-ray radiography" ], [ "plos", "/People and places/Population groupings/Professions/Medical personnel/Radiologists" ], [ "plos", "/Research and analysis methods/Imaging techniques/Diagnostic radiology/Bone imaging/X-ray radiography" ], [ "plos", "/Research and analysis methods/Imaging techniques/Diagnostic radiology/X-ray radiography" ], [ "plos", "/Research and analysis methods/Research design/Pilot studies" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Language" ], [ "plos", "/Social sciences/Sociology/Education" ] ], "openalex": "W7213451982", "pdf_url": "https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0001672&type=printable", "pmcid": null, "pmid": null, "published": "2026-09-17", "publisher": "Public Library of Science", "ris": "https://research.draconitic.com/paper/2bb88xbb0mbwt.ris", "sources": [ { "first_seen": "2026-10-30T12:17:49Z", "native_id": "10.1371/journal.pdig.0001672", "source": "plos", "url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001672" } ], "subjects": [ { "field": "medicine", "origin": "plos:diagnostic radiology", "subfield": "imaging" } ], "title": "Evaluating large language models for lay summaries of radiology reports using tailored prompting strategies and mixed-method assessment", "title_text": "Evaluating large language models for lay summaries of radiology reports using tailored prompting strategies and mixed-method assessment", "type": "peer-reviewed", "updated_at": "2026-10-30T12:17:49Z", "url": "https://research.draconitic.com/paper/2bb88xbb0mbwt", "venue": "PLOS Digital Health", "versions": [] }