{ "abstract": "Juvenile idiopathic arthritis (JIA) is the most common pediatric chronic inflammatory rheumatic condition, with substantial quality-of-life impacts. Treatment selection is complex due to limited guideline evidence, and difficult trade-offs. Decisions are often solely physician driven. We evaluated the feasibility of Large Language Model-based Decision Aid (LLM-DA) designed to support collaborative rather than solely physician decision-making in JIA care. A multi-method feasibility study was conducted in accordance with the TRIPOD-LLM checklist. The aims were twofold: first, to assess the quality of the LLM-DA (OpenAI’s o3 model) based on evaluations from three pediatric rheumatologists and an exploratory assessment by an LLM-as-a-Judge (OpenAI’s GPT-5 model); and second, to characterize perceived value of the tool from the patient perspective through a mixed methods evaluation (n = 34). For 10 de-identified use cases, rheumatologists and LLM-Judge generated treatment recommendations and scored the LLM-DA outputs on safety, factual consistency, treatment appropriateness, and preference acknowledgment. Rheumatologists rated 97% of outputs as acceptable for safety and preference acknowledgment, 60% for factual consistency, 70% for appropriateness. Treatment recommendation agreement among all evaluators and real-world data was low at <30%. Inter-rater agreement for scores among rheumatologists was almost perfect for safety and preference acknowledgement, moderate for treatment appropriateness, and poor for factual consistency, and remained the same or improved with LLM-as-a-Judge. Variability in evaluator scoring and treatment recommendations may reflect the inherent heterogeneity in acceptable JIA treatment pathways and potential limitations in the LLM-DA’s decision-making logic. Five patient themes were identified: the tool complements rheumatologist expertise, prepares patients through reflection and actionable information, builds confidence for self-advocacy, has variable value depending on the patient-physician relationship, and has potential for broader JIA self-management. These exploratory findings provide a proof-of-concept for using AI to support collaborative decision-making, while highlighting the need for future studies to determine whether LLMs can reliably support clinical practice. Author summary: Our work investigates the feasibility of using large language models (LLMs), to support treatment planning for juvenile idiopathic arthritis (JIA). JIA is hard to treat because evidence in clinical guidelines is low, so physicians often rely on their experiential knowledge to make decisions, and medications can be hard to manage due to harsh side effects. Given these challenges, there is a unique opportunity to investigate whether decision aids that use LLMs can help patients get more involved in their care. Our research provides preliminary evidence that the LLM-DA may support physician-mediated discussions, particularly by helping patients reflect on preferences and prepare for appointments. We also interviewed 34 individuals diagnosed with JIA and found that they thought the decision aid’s outputs were useful and felt more confident to be involved in the decision-making process. By helping young patients prepare, reflect on their preferences, and speak up during appointments, this tool shows potential to support more collaborative conversations.", "abstract_shown": true, "affiliations": [ "Institute of Health Policy, Management, and Evaluation, University of Toronto, Toronto, Ontario, Canada.", "Centre for Digital Therapeutics, University Health Network, Toronto, Ontario, Canada.", "Department of Pediatrics, Cumming School of Medicine, University of Calgary, Calgary, Albereta, Canada.", "Children's Health Ireland, and University College Dublin, Trinity College Dublin, Dublin, Ireland.", "Child Health Evaluative Sciences, Research Institute, The Hospital for Sick Children, Toronto, Ontario, Canada.", "Department of Computer Science, University of Toronto, Toronto, Ontario, Canada.", "Telfer School of Management, University of Ottawa, Ottawa, Ontario, Canada.", "Department of Pediatric Rheumatology and Immunology, Wilhelmina, Children's Hospital, University Medical Center Utrecht, Utrecht, The Netherlands.", "Faculty of Medicine, Utrecht University, Utrecht, The Netherlands.", "Center for Translational Immunology, University Medical Center Utrecht, Utrecht, The Netherlands.", "Department of Immunology and Medical Science, University of Toronto, Toronto, Ontario, Canada.", "Division of Rheumatology, The Hospital for Sick Children, Toronto, Ontario, Canada." ], "article_type": "Research Article", "arxiv": null, "author_total": 10, "authors": [ { "affs": [ 0, 1 ], "name": "Antonia Barbaric", "orcid": "0000-0002-8271-2030" }, { "affs": [ 2, 3 ], "name": "Susanne M Benseler", "orcid": "0000-0001-5756-3638" }, { "affs": [ 0, 4 ], "name": "Chitra Lalloo" }, { "affs": [ 0, 1 ], "name": "Xinyi Celine Liu" }, { "affs": [ 5 ], "name": "Alex Mariakakis", "orcid": "0000-0002-9986-3345" }, { "affs": [ 0, 1, 6 ], "name": "Quynh Pham" }, { "affs": [ 7, 8, 9 ], "name": "Sebastiaan J Vastert" }, { "affs": [ 10, 11 ], "name": "Rae S M Yeung" }, { "affs": [ 0, 1 ], "name": "Joseph A Cafazzo" }, { "affs": [ 7, 8 ], "name": "Joost F Swart" } ], "bibtex": "https://research.draconitic.com/paper/abvcg86re9apt.bib", "cited_by": 0, "doi": "10.1371/journal.pdig.0001655", "fulltext_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001655", "id": "abvcg86re9apt", "issns": [ "2767-3170" ], "landing_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001655", "language": "eng", "licence": "CC-BY", "licence_label": "CC BY", "licence_source": "article", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "lcc", "R858-859.7" ], [ "openalex", "2718|Artificial Intelligence in Healthcare and Education" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognition/Decision making" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Decision making" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Reasoning" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Decision making" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Reasoning" ], [ "plos", "/Medicine and health sciences/Health care/Health care policy/Treatment guidelines" ], [ "plos", "/Medicine and health sciences/Health care/Health care providers/Physicians" ], [ "plos", "/Medicine and health sciences/Pediatrics" ], [ "plos", "/Medicine and health sciences/Pharmaceutics/Drug therapy" ], [ "plos", "/Medicine and health sciences/Rheumatology" ], [ "plos", "/Medicine and health sciences/Rheumatology/Arthritis" ], [ "plos", "/People and places/Population groupings/Professions/Medical personnel/Physicians" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Decision making" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Reasoning" ] ], "openalex": "W7223243328", "pdf_url": "https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0001655&type=printable", "pmcid": "PMC13690242", "pmid": "42911979", "published": "2026-10-22", "publisher": "Public Library of Science", "ris": "https://research.draconitic.com/paper/abvcg86re9apt.ris", "sources": [ { "first_seen": "2026-10-30T12:17:23Z", "native_id": "10.1371/journal.pdig.0001655", "source": "plos", "url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001655" }, { "first_seen": "2026-10-30T12:20:10Z", "native_id": "MED:42911979", "source": "europepmc", "url": "https://europepmc.org/article/MED/42911979" }, { "first_seen": "2026-10-31T14:22:29Z", "native_id": "031e87f7b7b948528365848762bdf587", "source": "doaj", "url": "https://doi.org/10.1371/journal.pdig.0001655" } ], "subjects": [ { "field": "biology", "origin": "plos:neuroscience", "subfield": "neuroscience" } ], "title": "An LLM-based decision aid for collaborative decision-making in juvenile idiopathic arthritis: A multimethod evaluation", "title_text": "An LLM-based decision aid for collaborative decision-making in juvenile idiopathic arthritis: A multimethod evaluation", "type": "peer-reviewed", "updated_at": "2026-10-31T14:22:29Z", "url": "https://research.draconitic.com/paper/abvcg86re9apt", "venue": "PLOS Digital Health", "versions": [] }