{ "abstract": "Background\n\nPatients with infertility often have inadequate reproductive health literacy (RHL), while existing health education materials may be difficult to read, overly complex, and insufficiently tailored to individual needs. In recent years, large language models (LLMs) have created new opportunities to improve health education through their ability to generate and interpret medical information. However, their applicability and content quality in infertility education remain insufficiently evaluated.Objective\n\nThis study systematically evaluated the readability and content quality of LLM-generated health education materials for patients with infertility, identified model strengths and limitations, and provided evidence to support their standardized use in reproductive health.Methods\n\nThis study selected DeepSeek-R1, ChatGPT (GPT-5.2 Plus), Kimi 2.5, Qwen 3, and Gemini 3 as the study subjects. Based on the theoretical framework of RHL, this study developed a standardized question list comprising 6 dimensions and 93 items, and used unified prompts to generate health education texts in both Chinese and English. In this study, the Datayze Readability Analyzer was used to assess readability via the Flesch Reading Ease, Gunning Fog Index, and Flesch-Kincaid Grade Level (FKGL). Meanwhile, the content quality was evaluated in a double-blinded manner by reproductive medicine experts across 3 dimensions: accuracy, comprehensiveness, and humanistic care. Repeated-measures analysis of variance was used to compare performance across the LLMs, and Pearson correlation analysis was conducted to examine the associations between readability and content quality. A radar chart was used to evaluate the overall performance of each model.Results\n\nThe 5 LLMs differed significantly in both text readability and content quality. In terms of readability, Gemini 3 and Qwen 3 performed best, with significantly higher FRE scores than the other 3 models. DeepSeek-R1 and Kimi 2.5 performed least favorably, with FKGL scores corresponding to a ninth-grade reading level. ChatGPT (GPT-5.2 Plus) showed intermediate performance. In terms of content quality, Kimi 2.5 performed best across accuracy, comprehensiveness, and humanistic care, followed by DeepSeek-R1. ChatGPT (GPT-5.2 Plus) performed least favorably. Analysis revealed significant negative correlations between readability and content comprehensiveness for texts generated by Kimi 2.5, indicating a trade-off between these 2 attributes. The composite radar chart analysis showed that Kimi 2.5 achieved the highest score (11,594.33), whereas ChatGPT (GPT-5.2 Plus) had the lowest score (5555.83).Conclusions\n\nDifferent LLMs showed significant differences in performance in generating health education texts for patients with infertility. In some models, readability was negatively correlated with content comprehensiveness. Under the conditions of this study, Kimi 2.5 demonstrated strong overall performance. However, no model achieved the best performance across all evaluation dimensions. Future applications of LLMs in reproductive health education will require domain-specific optimization, context-specific adaptation, and systematic quality control to support more accurate, comprehensive, and patient-centered communication.", "abstract_shown": true, "affiliations": [ "Changzhou Third People's Hospital, Changzhou Medical Center, Nanjing Medical University, Changzhou, Jiangsu, China.", "Department of Reproduction, Changzhou Maternal and Child Health Care Hospital, Changzhou Medical Center, Nanjing Medical University, No 16 Dingxiang Road, Zhonglou District, Changzhou, Jiangsu, China, 86 (0519) 88581188.", "School of Public Health, Xuzhou Medical University, Xuzhou, Jiangsu, China.", "Tianjin University of Traditional Chinese Medicine, Tianjin, China.", "The Affiliated Changzhou Hospital of Xuzhou Medical University, Changzhou Third People's Hospital, Changzhou, Jiangsu, China." ], "article_type": "research article", "arxiv": null, "author_total": 7, "authors": [ { "affs": [ 0, 1 ], "name": "Renjie Lu", "orcid": "0000-0003-2106-8277" }, { "affs": [ 2 ], "name": "Yuchen Wei", "orcid": "0009-0001-0782-0007" }, { "affs": [ 3 ], "name": "Siyuan Wu", "orcid": "0009-0001-7298-1675" }, { "affs": [ 2 ], "name": "Yaning Han", "orcid": "0009-0005-7311-8389" }, { "affs": [ 4 ], "name": "Shenyu Zhao", "orcid": "0000-0003-2731-9831" }, { "affs": [ 1 ], "name": "Lingmin Hu", "orcid": "0000-0002-7374-9106" }, { "affs": [ 1 ], "name": "Yang Chen", "orcid": "0009-0008-4516-4646" } ], "bibtex": "https://research.draconitic.com/paper/fw4pp1nw1jq2y.bib", "cited_by": 0, "doi": "10.2196/97878", "fulltext_url": "https://europepmc.org/articles/PMC13727392", "id": "fw4pp1nw1jq2y", "issns": [ "1439-4456", "1438-8871" ], "landing_url": "https://europepmc.org/article/MED/42971940", "language": "eng", "licence": "CC-BY", "licence_label": "CC BY", "licence_source": "article", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "heading", "Information and Patient Education on Healthy Aging" ], [ "heading", "Machine Learning" ], [ "heading", "Natural Language Processing" ], [ "heading", "New Resources for Medical Education" ], [ "heading", "Original Paper" ], [ "heading", "eHealth Literacy / Digital Literacy" ], [ "openalex", "2718|Artificial Intelligence in Healthcare and Education" ] ], "openalex": "W7225154086", "pdf_url": "https://europepmc.org/articles/PMC13727392?pdf=render", "pmcid": "PMC13727392", "pmid": "42971940", "published": "2026-10-27", "publisher": "JMIR Publications Inc.", "ris": "https://research.draconitic.com/paper/fw4pp1nw1jq2y.ris", "sources": [ { "first_seen": "2026-11-04T15:20:44Z", "native_id": "MED:42971940", "source": "europepmc", "url": "https://europepmc.org/article/MED/42971940" }, { "first_seen": "2026-11-06T12:23:16Z", "native_id": "PMC13727392", "source": "pmc", "url": "https://doi.org/10.2196/97878" } ], "subjects": [ { "field": "medicine", "origin": "openalex:2718|Artificial Intelligence in Healthcare and Education", "subfield": "epidemiology-public-health" } ], "title": "Evaluating the Performance of Five Leading Large Language Models in Generating Health Education Materials for Patients With Infertility: Comparative Analysis.", "title_text": "Evaluating the Performance of Five Leading Large Language Models in Generating Health Education Materials for Patients With Infertility: Comparative Analysis.", "type": "peer-reviewed", "updated_at": "2026-11-07T12:20:35Z", "url": "https://research.draconitic.com/paper/fw4pp1nw1jq2y", "venue": "Journal of medical Internet research", "versions": [] }