Trigger-first large language model-assisted adverse drug reaction surveillance: A pharmacist-supervised retrospective cohort study
- 1School of Electromechanical Engineering, Guangdong University of Technology, Guangzhou, Guangdong, 510006, China
- 2Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, Guangdong, 510120, China
- 3Sun Yat-sen Memorial Hospital, Sun Yat-sen University, Guangzhou, Guangdong, 510120, China; Corresponding author.
Abstract
Background: Active pharmacovigilance in low-prevalence settings requires review of large electronic health record cohorts. We evaluated a pharmacist-supervised workflow combining electronic triggers with large language model (LLM)-assisted review. Objective: To assess candidate enrichment and stage-specific performance for adverse drug reaction (ADR) surveillance. Methods: In this retrospective cohort study, two pharmacists established a blinded reference standard for 7950 exposure episodes from 7950 unique inpatients. A prespecified Chinese Hospital Pharmacovigilance System trigger selected candidates for Qwen3-4B-Instruct and an institution-specific low-rank adaptation (LoRA)-fine-tuned model trained on 410 patient-disjoint cases involving other medicines. Paired errors were compared using exact McNemar tests. Results: Twenty-eight episodes were ADR-positive. The trigger selected 167 candidates (2.10%), with 100.0% sensitivity (95% confidence interval, 87.66–100.0%) and 98.25% specificity. For the base versus LoRA-fine-tuned model, sensitivity was 82.14% versus 92.86%, specificity 88.49% versus 94.96%, positive predictive value 58.97% versus 78.79%, and F1 score 68.66% versus 85.25%. LoRA reduced false positives from 16 to 7 (9 corrected, 0 new; P = 0.0039). The numerical increase in sensitivity was not statistically significant ( P = 0.250), with only 28 ADR-positive episodes available for evaluation. Conclusions: Trigger-first screening reduced review volume, while LoRA fine-tuning improved false-positive control. LLM classifications should support pharmacist review and should not be used as an autonomous exclusion rule.
Related
- A Structured, Auditable Taxonomy for Pharmacist-Identifiable Medication-Related Risk Signals in Routine Admission Data: A Retrospective Medication Safety Audit in a High-Risk Inpatient Cohort
- Intensive pharmacist-led guideline-directed medical therapy for hospitalized heart failure patients: a single-center retrospective cohort study.
- Respiratory pathogen epidemiology in children under 5 years hospitalized with acute respiratory infections in Qingdao, China: a retrospective, single-center, repeated cross-sectional surveillance study
- A Six-Year Retrospective Surveillance of Outpatient Staphylococcus aureus Resistance Dynamics in Skin and Soft Tissue Infections: Are Non-β-Lactam Options Still Reliable?
- Rising active surveillance/observation rates and regimen intensification in young men: A retrospective cohort study of national commercial health insurance claims data
BibTeX
@article{liao2026triggerfirst,
title = {{Trigger-first large language model-assisted adverse drug reaction surveillance: A pharmacist-supervised retrospective cohort study}},
author = {Canhui Liao and Xinjun Lai and Wenge Chen and Jianxiong He and Xiaoxia Yu},
journal = {Exploratory Research in Clinical and Social Pharmacy},
year = {2026},
month = oct,
doi = {10.1016/j.rcsop.2026.100866},
url = {https://doi.org/10.1016/j.rcsop.2026.100866},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - Trigger-first large language model-assisted adverse drug reaction surveillance: A pharmacist-supervised retrospective cohort study
AU - Canhui Liao
AU - Xinjun Lai
AU - Wenge Chen
AU - Jianxiong He
AU - Xiaoxia Yu
PY - 2026
DA - 2026/10/20
JO - Exploratory Research in Clinical and Social Pharmacy
DO - 10.1016/j.rcsop.2026.100866
UR - https://doi.org/10.1016/j.rcsop.2026.100866
AB - Background: Active pharmacovigilance in low-prevalence settings requires review of large electronic health record cohorts. We evaluated a pharmacist-supervised workflow combining electronic triggers with large language model (LLM)-assisted review. Objective: To assess candidate enrichment and stage-specific performance for adverse drug reaction (ADR) surveillance. Methods: In this retrospective cohort study, two pharmacists established a blinded reference standard for 7950 exposure episodes from 7950 unique inpatients. A prespecified Chinese Hospital Pharmacovigilance System trigger selected candidates for Qwen3-4B-Instruct and an institution-specific low-rank adaptation (LoRA)-fine-tuned model trained on 410 patient-disjoint cases involving other medicines. Paired errors were compared using exact McNemar tests. Results: Twenty-eight episodes were ADR-positive. The trigger selected 167 candidates (2.10%), with 100.0% sensitivity (95% confidence interval, 87.66–100.0%) and 98.25% specificity. For the base versus LoRA-fine-tuned model, sensitivity was 82.14% versus 92.86%, specificity 88.49% versus 94.96%, positive predictive value 58.97% versus 78.79%, and F1 score 68.66% versus 85.25%. LoRA reduced false positives from 16 to 7 (9 corrected, 0 new; P = 0.0039). The numerical increase in sensitivity was not statistically significant ( P = 0.250), with only 28 ADR-positive episodes available for evaluation. Conclusions: Trigger-first screening reduced review volume, while LoRA fine-tuning improved false-positive control. LLM classifications should support pharmacist review and should not be used as an autonomous exclusion rule.
SN - 2667-2766
ER -