{ "abstract": "Large language models (LLMs) have shown incredible promise in medicine. While LLMs may be particularly useful in areas requiring extensive review of clinical records, their use remains limited due to their tendency to hallucinate and fabricate information. Hallucination issues, as well as their consequences, are exacerbated in low–probability, high–stakes scenarios such as rare adverse safety events or medical errors. We present SAFE–AI (Structured and Automated Framework for Explainable AI), a novel method for clinical decision making that combines the strengths of clinical expert knowledge with LLMs in an ontology–driven model that minimizes hallucinations using strict rules. We test this method to identify medication errors in medical charts. We collected a sample of 18,402 lines of clinical information from 300 EMS clinical charts that were independently dually reviewed by two expert physicians for epinephrine adverse safety events (ASEs), with 96% inter-rater agreement. We tested SAFE–AI against these labels, achieving similar performance to human experts in detecting epinephrine overdoses with 97.9% accuracy, and 91.6% accuracy in identifying delays in epinephrine administration, greatly outperforming baseline LLMs models. Notably, some disagreements between clinicians and the model were found to be justifiable differences in judgment rather than errors. SAFE-AI presents a novel approach for clinical AI applications that addresses two key limitations of current machine learning methods: 1) over-reliance on probabilistic pattern recognition instead of established medical knowledge, and 2) perpetuation of biases present in training data. This framework is easily adaptable to a range of clinical applications, paving the way for provable and trustworthy AI in medicine. Author summary: LLMs have shown promise in analyzing clinical records but their use is limited due to their tendency to hallucinate and fabricate information. Misinformation could threaten patient safety and jeopardize trust. We developed SAFE–AI (Structured and Automated Framework for Explainable AI), which combines knowledge from clinical experts with LLM inference to detect adverse safety events (ASEs) with minimal errors. We tested our method in identifying medication errors in medical charts and compared results to reviews by expert physicians. Our method detected epinephrine delays and overdoses with a high level of accuracy. SAFE-AI presents a novel approach for clinical AI applications that overcomes reliance on pattern recognition instead of medical knowledge biases present in training data.", "abstract_shown": true, "affiliations": [ "Center for Learning Health Care Delivery, Beth Israel Deaconess Medical Center, Boston, Massachusetts, United States of America", "Department of Obstetrics Gynecology and Reproductive Medicine, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, Massachusetts, United States of America", "McWilliams School of Biomedical Informatics, UTHealth, Houston, Texas, United States of America", "Department of Emergency Medicine, Oregon Health and Science University, Portland, Oregon, United States of America", "Department of Biomedical Informatics, Harvard Medical School, Boston, Massachusetts, United States of America", "Department of Pediatrics and Emergency Medicine, University of British Columbia, Vancouver, Canada", "Department of Pediatrics, Oregon Health and Science University, Portland, Oregon, United States of America", "Department of Medical Informatics and Clinical Epidemiology, Oregon Health and Science University, Portland, Oregon, United States of America" ], "article_type": "Research Article", "arxiv": null, "author_total": 12, "authors": [ { "affs": [ 0 ], "name": "Diego Trujillo" }, { "affs": [ 1 ], "name": "Dulin Wang" }, { "affs": [ 2 ], "name": "Nathan Bahr" }, { "affs": [ 3 ], "name": "Tina Yi-Jin Hsieh" }, { "affs": [ 4 ], "name": "Byeongyeon Cho" }, { "affs": [ 5 ], "name": "Garth Meckler" }, { "affs": [ 6 ], "name": "Matthew Hansen" }, { "affs": [ 7 ], "name": "Carl Eriksson" }, { "name": "Kyu Seo Kim" }, { "name": "Steven Bedrick" }, { "name": "Xiaoqian Jiang" }, { "name": "Jeanne-Marie Guise" } ], "bibtex": "https://research.draconitic.com/paper/2h4ph9a8v3y9s.bib", "cited_by": 3, "doi": "10.1371/journal.pdig.0001174", "fulltext_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001174", "id": "2h4ph9a8v3y9s", "issns": [ "2767-3170" ], "landing_url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001174", "language": null, "licence": "CC-BY-4.0", "licence_label": "CC BY 4.0", "licence_source": "journal", "licence_url": "https://creativecommons.org/licenses/by/4.0/", "metadata_licence": null, "native_subjects": [ [ "openalex", "1702|Machine Learning in Healthcare" ], [ "plos", "/Biology and life sciences/Biochemistry/Hormones/Catecholamines/Epinephrine" ], [ "plos", "/Biology and life sciences/Biochemistry/Neurochemistry/Neurotransmitters/Biogenic amines/Catecholamines/Epinephrine" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognition/Decision making" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Decision making" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Perception/Sensory perception/Hallucinations" ], [ "plos", "/Biology and life sciences/Neuroscience/Cognitive science/Cognitive psychology/Reasoning" ], [ "plos", "/Biology and life sciences/Neuroscience/Neurochemistry/Neurotransmitters/Biogenic amines/Catecholamines/Epinephrine" ], [ "plos", "/Biology and life sciences/Neuroscience/Sensory perception/Hallucinations" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Decision making" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Perception/Sensory perception/Hallucinations" ], [ "plos", "/Biology and life sciences/Psychology/Cognitive psychology/Reasoning" ], [ "plos", "/Computer and information sciences/Artificial intelligence" ], [ "plos", "/Computer and information sciences/Data management/Data visualization/Infographics/Charts" ], [ "plos", "/Medicine and health sciences/Critical care and emergency medicine" ], [ "plos", "/Medicine and health sciences/Pediatrics" ], [ "plos", "/Physical sciences/Chemistry/Chemical compounds/Organic compounds/Amines/Catecholamines/Epinephrine" ], [ "plos", "/Physical sciences/Chemistry/Organic chemistry/Organic compounds/Amines/Catecholamines/Epinephrine" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Decision making" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Perception/Sensory perception/Hallucinations" ], [ "plos", "/Social sciences/Psychology/Cognitive psychology/Reasoning" ] ], "openalex": "W7213451977", "pdf_url": "https://journals.plos.org/digitalhealth/article/file?id=10.1371/journal.pdig.0001174&type=printable", "pmcid": null, "pmid": null, "published": "2026-09-17", "publisher": "Public Library of Science", "ris": "https://research.draconitic.com/paper/2h4ph9a8v3y9s.ris", "sources": [ { "first_seen": "2026-10-30T12:17:49Z", "native_id": "10.1371/journal.pdig.0001174", "source": "plos", "url": "https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001174" } ], "subjects": [ { "field": "biology", "origin": "plos:neuroscience", "subfield": "neuroscience" } ], "title": "A medically grounded LLM agent–based tool to detect patient safety events in medical records", "title_text": "A medically grounded LLM agent–based tool to detect patient safety events in medical records", "type": "peer-reviewed", "updated_at": "2026-10-30T12:17:49Z", "url": "https://research.draconitic.com/paper/2h4ph9a8v3y9s", "venue": "PLOS Digital Health", "versions": [] }