Skip to content
DraconiticResearch
Search Sources API
226,742 studies31 sourcesUpdated 43 min ago
Peer-reviewedCC BY-NC 4.0BMJ Health & Care Informatics

Automatic extraction of multiple contributing factors in medication-related incidents based on the P-mSHELL model: development and evaluation of a BERT-based multilabel classifier

4 affiliations
  1. 1Keio University Faculty of Pharmacy, Minato-ku, Tokyo, Japan
  2. 2Institute of Library, Information and Media Science, University of Tsukuba, Tsukuba, Ibaraki, Japan
  3. 3Nara Institute of Science and Technology, Ikoma, Nara, Japan
  4. 4Kameda General Hospital, Kamogawa, Chiba, Japan

Abstract

Objectives: Manual analysis of contributing factors in free-text incident narratives is labour intensive. We aimed to develop and internally validate a multilabel natural language processing (NLP) classifier that extracts multiple contributing factors from medication-related incident reports using the P-mSHELL human factors framework.

Methods: We analysed 5058 sentences from 1192 medication-related incident reports (2020–2022) from the Japan Council for Quality Health Care Medical Accident Information Collection Project. Eight contributing factor categories were defined based on a modified P-mSHELL model: Medical Professional (MP), Patient Family (PF), Patient (P), Management (M), Software (S), Hardware (H), Environment (E) and Liveware-Liveware (L-L). Inter-annotator agreement was assessed using Cohen’s kappa (κ) on a random subset. A multilabel classifier was developed by fine-tuning a pretrained Japanese Bidirectional Encoder Representations from Transformers (BERT) model and evaluated using report-level fivefold cross-validation, F1 scores and exact match accuracy.

Results: Substantial inter-annotator agreement (κ=0.676–1.000) was observed across all categories. Multiple labels were assigned to 67.0% of sentences, with Medical Professional (62.9%) and Software (59.9%) most frequent. The classifier achieved F1 scores from 0.490 (Hardware) to 0.902 (Medical Professional), with sentence-level exact match accuracy of 0.512 (95% CI 0.496 to 0.527).

Discussion: The classifier performed well for major factor categories, although its accuracy does not yet support fully automated analysis; pending external validation, it is best positioned as a decision support tool to assist—rather than replace—expert review of incident reports.

Conclusion: This study demonstrates the feasibility of NLP-based multilabel extraction of contributing factors within the P-mSHELL framework, supporting scalable and consistent medication-safety incident analysis.

Related

  1. Development and psychometric validation of a barrier assessment tool for adherence to prescribed medication regimens in post-PCI patients
    Anit Kumar Sinha, Pandey Pratima, Pooja Arora +3
    Future CardiologyPeer-reviewedLicence not statedFull textCardiovascular medicinePMC
  2. Development of a nomogram for identifying prevalent medication discrepancy in maintenance hemodialysis patients: a multicenter cross-sectional study
    Xiaodan Lin, Mianzhou Liu, Zhaochuan Li +7
    Frontiers in medicinePeer-reviewedCC BYFull textGeneral medicineEurope PMC
  3. A precision health approach to medication management in neurodevelopmental conditions: a model development and validation study using four international cohorts
    Marlee M. Vandewouw, Kamran Niroomand, Harshit Bokadia +15
    Translational PsychiatryPeer-reviewedCC BY 4.0Psychiatry & mental healthCrossref
  4. Process evaluation of the INTENSE study aimed at improving adherence to oral diabetes and/or hypertension medication in people with type 2 diabetes
    M. Langendoen-Gort, D. Bhattacharya, H. Al-Jabr +5
    medRxivPreprintAll rights reservedFull textGeneral medicinemedRxiv
  5. GLLaucoMed: A Secure Large Language Model-Powered Agentic Workflow for Automated Medication Extraction from Free-Text Glaucoma Clinical Notes.
    Nicholas Solages, Rafael Scherer, Gustavo A Samico +4
    Ophthalmology sciencePeer-reviewedCC BY-NC-NDFull textGeneral medicineEurope PMC
BibTeX
@article{ebara2026automatic,
  title = {{Automatic extraction of multiple contributing factors in medication-related incidents based on the P-mSHELL model: development and evaluation of a BERT-based multilabel classifier}},
  author = {Sayaka Ebara and Hayato Kizaki and Yuri Takamatsu and Satoshi Watabe and Shungo Imai and Shuntaro Yada and Eiji Aramaki and Osamu Yasumuro and Ryohkan Funakoshi and Satoko Hori},
  journal = {BMJ Health \& Care Informatics},
  year = {2026},
  month = nov,
  doi = {10.1136/bmjhci-2026-102260},
  url = {https://doi.org/10.1136/bmjhci-2026-102260},
  note = {CC BY-NC 4.0},
}
RIS
TY  - JOUR
TI  - Automatic extraction of multiple contributing factors in medication-related incidents based on the P-mSHELL model: development and evaluation of a BERT-based multilabel classifier
AU  - Sayaka Ebara
AU  - Hayato Kizaki
AU  - Yuri Takamatsu
AU  - Satoshi Watabe
AU  - Shungo Imai
AU  - Shuntaro Yada
AU  - Eiji Aramaki
AU  - Osamu Yasumuro
AU  - Ryohkan Funakoshi
AU  - Satoko Hori
PY  - 2026
DA  - 2026/11/11
JO  - BMJ Health & Care Informatics
DO  - 10.1136/bmjhci-2026-102260
UR  - https://doi.org/10.1136/bmjhci-2026-102260
AB  - Objectives: Manual analysis of contributing factors in free-text incident narratives is labour intensive. We aimed to develop and internally validate a multilabel natural language processing (NLP) classifier that extracts multiple contributing factors from medication-related incident reports using the P-mSHELL human factors framework.  Methods: We analysed 5058 sentences from 1192 medication-related incident reports (2020–2022) from the Japan Council for Quality Health Care Medical Accident Information Collection Project. Eight contributing factor categories were defined based on a modified P-mSHELL model: Medical Professional (MP), Patient Family (PF), Patient (P), Management (M), Software (S), Hardware (H), Environment (E) and Liveware-Liveware (L-L). Inter-annotator agreement was assessed using Cohen’s kappa (κ) on a random subset. A multilabel classifier was developed by fine-tuning a pretrained Japanese Bidirectional Encoder Representations from Transformers (BERT) model and evaluated using report-level fivefold cross-validation, F1 scores and exact match accuracy.  Results: Substantial inter-annotator agreement (κ=0.676–1.000) was observed across all categories. Multiple labels were assigned to 67.0% of sentences, with Medical Professional (62.9%) and Software (59.9%) most frequent. The classifier achieved F1 scores from 0.490 (Hardware) to 0.902 (Medical Professional), with sentence-level exact match accuracy of 0.512 (95% CI 0.496 to 0.527).  Discussion: The classifier performed well for major factor categories, although its accuracy does not yet support fully automated analysis; pending external validation, it is best positioned as a decision support tool to assist—rather than replace—expert review of incident reports.  Conclusion: This study demonstrates the feasibility of NLP-based multilabel extraction of contributing factors within the P-mSHELL framework, supporting scalable and consistent medication-safety incident analysis.
SN  - 2632-1009
ER  -