Multi-Agent Generative Artificial Intelligence for Transfusion Reaction Feature Extraction
- 1Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada
Abstract
BACKGROUND: Many transfusion reactions are not reported to hemovigilance systems, and the features needed to recognize them are often documented only in free-text clinical notes. We sought to develop and evaluate an open-source, multi-agent artificial intelligence (AI) pipeline to extract transfusion reaction features defined by the National Healthcare Safety Network (NHSN) Hemovigilance Module from clinical notes. METHODS: We curated 181 features from the NHSN protocol and developed a multi-stage pipeline that extracted features, their timing, and clinical details from free-text notes. Three open-weight large language models (Qwen 3.6-27B, GPT-OSS-120B, and Kimi K3) were applied to 50 notes from expert-written scenarios, published case reports, and MIMIC-IV discharge summaries. Two physicians independently evaluated every model output in a blinded interface. The primary analysis was restricted to feature occurrences with concordant physician assessments. RESULTS: Physicians agreed on 1,201 of 1,343 co-rated feature occurrences (89.4%; Krippendorff's alpha 0.73). Precision was above 0.98 for all models. Sensitivity was 0.774 (95% CI 0.745-0.800) for Kimi K3, 0.636 (0.604-0.667) for GPT-OSS-120B, and 0.624 (0.593-0.655) for Qwen 3.6-27B, and specificity ranged from 0.952 to 0.991. Omissions accounted for more than 94% of false negatives. Timing and clinical details were judged supported or reasonable in more than 92% and 97% of assessments, respectively. Results were consistent across note sources and in analyses including discordant assessments. CONCLUSION: Multi-agent AI pipelines provide a feasible method for extracting transfusion reaction signals from clinical notes with high precision. Our work may have applications in active hemovigilance, reaction classification, and big data research.
Related
- Generative Artificial Intelligence Models in Breast Cancer Diagnosis: A Systematic Review
- Nursing students’ experiences with using generative artificial intelligence: a descriptive qualitative study
- Generative Artificial Intelligence in Periodontology: Application of OpenAI Models for the Classification of Periodontitis and Risk Assessment in Patients Undergoing Supportive Periodontal Care
- Simulating Psychotherapeutic Sessions with Generative Artificial Intelligence: A Proof-of-Concept Study of In Silico Psychotherapy Research
- Multidimensional Assurance of an Electronic Health Record Embedded Generative Artificial Intelligence Summarization Tool
BibTeX
@misc{kumari2026multiagent,
title = {{Multi-Agent Generative Artificial Intelligence for Transfusion Reaction Feature Extraction}},
author = {A. Kumari and J. W. Jacobs and L. Dilly and A. Anand and J. Woo and Y. Lin and A. W. Shih and G. S. Booth and I. Raslan and N. Relke and R. Dubey and S. Srivastava and W. Abdulwahab and R. Goel and S. Raza},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.06.26367389},
url = {https://www.medrxiv.org/content/10.64898/2026.11.06.26367389v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Multi-Agent Generative Artificial Intelligence for Transfusion Reaction Feature Extraction
AU - A. Kumari
AU - J. W. Jacobs
AU - L. Dilly
AU - A. Anand
AU - J. Woo
AU - Y. Lin
AU - A. W. Shih
AU - G. S. Booth
AU - I. Raslan
AU - N. Relke
AU - R. Dubey
AU - S. Srivastava
AU - W. Abdulwahab
AU - R. Goel
AU - S. Raza
PY - 2026
DA - 2026/11/06
PB - medRxiv
DO - 10.64898/2026.11.06.26367389
UR - https://www.medrxiv.org/content/10.64898/2026.11.06.26367389v1
AB - BACKGROUND: Many transfusion reactions are not reported to hemovigilance systems, and the features needed to recognize them are often documented only in free-text clinical notes. We sought to develop and evaluate an open-source, multi-agent artificial intelligence (AI) pipeline to extract transfusion reaction features defined by the National Healthcare Safety Network (NHSN) Hemovigilance Module from clinical notes. METHODS: We curated 181 features from the NHSN protocol and developed a multi-stage pipeline that extracted features, their timing, and clinical details from free-text notes. Three open-weight large language models (Qwen 3.6-27B, GPT-OSS-120B, and Kimi K3) were applied to 50 notes from expert-written scenarios, published case reports, and MIMIC-IV discharge summaries. Two physicians independently evaluated every model output in a blinded interface. The primary analysis was restricted to feature occurrences with concordant physician assessments. RESULTS: Physicians agreed on 1,201 of 1,343 co-rated feature occurrences (89.4%; Krippendorff's alpha 0.73). Precision was above 0.98 for all models. Sensitivity was 0.774 (95% CI 0.745-0.800) for Kimi K3, 0.636 (0.604-0.667) for GPT-OSS-120B, and 0.624 (0.593-0.655) for Qwen 3.6-27B, and specificity ranged from 0.952 to 0.991. Omissions accounted for more than 94% of false negatives. Timing and clinical details were judged supported or reasonable in more than 92% and 97% of assessments, respectively. Results were consistent across note sources and in analyses including discordant assessments. CONCLUSION: Multi-agent AI pipelines provide a feasible method for extracting transfusion reaction signals from clinical notes with high precision. Our work may have applications in active hemovigilance, reaction classification, and big data research.
ER -