Skip to content
DraconiticResearch
Search Sources API
203,952 studies31 sourcesUpdated 16 min ago
Peer-reviewedCC BY 4.0NPJ Digital Medicine

Automated risk scoring for venous thromboembolism using large language models with expert knowledge-augmented prompting: a multicenter validation study

  • Jing Ma1
  • Dingyi Wang2
  • Yaqian Zhang3
  • Zhaofei Chen4
  • Kaiyuan Zhen5
  • Mengyao Li6,7
  • Jingrui Li6,7
  • Yunfan Shi8
  • Xiaomeng Zhang9
  • Rui Liang10
  • Feiya Xu11
  • Siqi Ma12
17 more authors
  • Jintao Chen13
  • Yanshuang Lyu14
  • Yuanhua Yang15
  • Zhe Cheng16
  • Yutao Guo17
  • Lei Xia18
  • Wei Wang19
  • Bing Liu20
  • Cunbo Jia21
  • Shuai Zhang7
  • Qian Gao7
  • Yunxia Zhang7
  • Zhu Zhang7
  • Wanmu Xie7
  • Guohui Fan2
  • Shengfeng Wang20,22
  • Zhenguo Zhai7,23
23 affiliations
  1. 1Department of Respiratory and Critical Care Medicine, The Second Affiliated Hospital of Harbin Medical University, Harbin, China
  2. 2National Center for Respiratory Medicine; State Key Laboratory of Respiratory Health and Multimorbidity; National Clinical Research Center for Respiratory Diseases; Institute of Respiratory Medicine, Chinese Academy of Medical Sciences & Peking Union Medical College; Department of Clinical research and Data management, Center of Respiratory Medicine, China-Japan Friendship Hospital, Beijing, China
  3. 3Department of Respiratory and Critical Care Medicine, China-Japan Friendship Institute of Clinical Medicine, Beijing, China
  4. 4China-Japan Friendship Hospital, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
  5. 5Respiratory Department, Center for Pulmonary Vascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College, State Key Laboratory of Cardiovascular Disease, National Center for Cardiovascular Diseases, Beijing, China
  6. 6Peking University China-Japan Friendship School of Clinical Medicine, Beijing, China
  7. 7National Center for Respiratory Medicine; State Key Laboratory of Respiratory Health and Multimorbidity; National Clinical Research Center for Respiratory Diseases; Institute of Respiratory Medicine, Chinese Academy of Medical Sciences & Peking Union Medical College; Department of Pulmonary and Critical Care Medicine, Center of Respiratory Medicine, China-Japan Friendship Hospital, Beijing, China
  8. 8Capital Medical University, Beijing, China
  9. 9Peking University China-Japan Friendship School of Clinical Medicine; Department of Pulmonary and Critical Care Medicine; Center of Respiratory Medicine; National Center for Respiratory Medicine; Institute of Respiratory Medicine, Chinese Academy of Medical Sciences; National Clinical Research Center for Respiratory Diseases, China-Japan Friendship Hospital, Beijing, China
  10. 10Beijing University of Chinese Medicine, Beijing, China
  11. 11China-Japan Friendship Hospital (Institute of Clinical Medical Sciences), Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
  12. 12BioMap Co., Ltd, Beijing, China
  13. 13Zhejiang University, Hangzhou, China
  14. 14Department of Physiology, Institute of Basic Medical Sciences, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China
  15. 15Department of Pulmonary and Critical Care Medicine, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China
  16. 16Department of Pulmonary and Critical Care Medicine, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China
  17. 17Department of Pulmonary Vessel and Thrombotic Disease, Sixth Medical Center, Chinese PLA General Hospital, Beijing, China
  18. 18Medical Affairs Department of China-Japan Friendship Hospital, Beijing, China
  19. 19Department of Nursing, China-Japan Friendship Hospital, Beijing, China
  20. 20Key Laboratory of Epidemiology of Major Diseases (Peking University), Ministry of Education, Beijing, China
  21. 21China-Japan Friendship Hospital, Beijing, China
  22. 22Department of Epidemiology and Biostatistics, School of Public Health, Peking University, Beijing, China
  23. 23School of Population Medicine and Public Health, Chinese Academy of Medical Sciences & Peking Union Medical College; State Key Laboratory of Respiratory Health and Multimorbidity; Key Laboratory of Pathogen Infection Prevention and Control (Peking Union Medical College), Ministry of Education, Beijing, China
Read full textPDFDOICite

Abstract

Venous thromboembolism (VTE) is a common but preventable complication in hospitalized patients, yet standardized risk scoring using unstructured electronic health records (EHRs) remains challenging. We retrospectively analyzed anonymized EHRs from a nationwide multicenter cohort across 30 hospitals. Using stratified random sampling, 50 cases from 10 hospitals were used for development and 200 cases from the remaining 20 hospitals for testing. Expert knowledge-augmented prompts for the Padua and Caprini scores were developed through a Delphi process involving 19 specialists and compared with basic and complex prompts. Performance was evaluated using six open-source large language models (LLMs), with prevalence-adjusted bias-adjusted kappa (PABAK) and F1 score as primary metrics. Expert knowledge-augmented prompts showed the highest performance in both datasets. In the test dataset, mean item-level PABAK and F1 scores were 0.97 and 0.96 for Padua, and 0.97 and 0.93 for Caprini; most items showed PABAK and F1 scores above 0.90. Risk stratification was better for Padua (PABAK 0.92; F1 score 0.96) than for Caprini (PABAK 0.73; F1 score 0.64), with processing times of 6.07 s and 12.82 s per case, respectively. These findings indicated that LLMs with expert knowledge-augmented prompting could efficiently support automated Padua and Caprini scoring and risk stratification in thrombosis prevention workflows.

Related

  1. KGRD: a knowledge-graph-augmented automated reasoning framework for diagnosis and counselling of paediatric rare genetic disorders
    Guiling Guo, Zhenyu Shao, Heying Luo +8
    NPJ Digital MedicinePeer-reviewedCC BY-NC-ND 4.0Full textEpidemiology & public healthPMC
  2. CO090 | DIRECT ORAL ANTICOAGULANTS IN THE SECONDARY PREVENTION OF VENOUS THROMBOEMBOLISM IN PATIENTS WITH SEVERE INHERITED THROMBOPHILIA: A REAL-WORLD MULTICENTER STUDY
    Antonella Tufano
    Bleeding, Thrombosis and Vascular BiologyPeer-reviewedCC BY-NCFull textCardiovascular medicineDOAJ
  3. Graph-Enhanced Retrieval-Augmented Generation for Postoperative Patient Question Answering: A Comparative Evaluation of Knowledge Graph, Retrieval-Augmented Generation, and Hybrid GraphRAG Approaches
    Srinivasagam Prabha, Bernardo G. Collaco, Carina Rosa Malena +3
    Bioengineering (Basel, Switzerland)Peer-reviewedCC BYFull textEurope PMC
  4. MSAT-KOA-Net: a multi-scale attention transformer network with retrieval-augmented knowledge (RAG) for interpretable knee osteoarthritis grading
    Shaowei Jiang, Bin Xu, K. Venkatachalam +1
    Scientific ReportsPeer-reviewedCC BY-NC-ND 4.0Full textCrossref
  5. Knowledge, attitudes, and practices regarding venous thromboembolism prevention in patients with lung cancer
    Yuhang Ding, Yanhong Meng, Pengnan Xu +2
    Frontiers in Public HealthPeer-reviewedCC BY 4.0Full textEpidemiology & public healthCrossrefPMC

Cite

BibTeXRIS
BibTeX
@article{ma2026automated,
  title = {{Automated risk scoring for venous thromboembolism using large language models with expert knowledge-augmented prompting: a multicenter validation study}},
  author = {Jing Ma and Dingyi Wang and Yaqian Zhang and Zhaofei Chen and Kaiyuan Zhen and Mengyao Li and Jingrui Li and Yunfan Shi and Xiaomeng Zhang and Rui Liang and Feiya Xu and Siqi Ma and Jintao Chen and Yanshuang Lyu and Yuanhua Yang and Zhe Cheng and Yutao Guo and Lei Xia and Wei Wang and Bing Liu and Cunbo Jia and Shuai Zhang and Qian Gao and Yunxia Zhang and Zhu Zhang and Wanmu Xie and Guohui Fan and Shengfeng Wang and Zhenguo Zhai},
  journal = {NPJ Digital Medicine},
  year = {2026},
  month = jul,
  doi = {10.1038/s41746-026-02929-3},
  url = {https://doi.org/10.1038/s41746-026-02929-3},
  note = {CC BY 4.0},
}
RIS
TY  - JOUR
TI  - Automated risk scoring for venous thromboembolism using large language models with expert knowledge-augmented prompting: a multicenter validation study
AU  - Jing Ma
AU  - Dingyi Wang
AU  - Yaqian Zhang
AU  - Zhaofei Chen
AU  - Kaiyuan Zhen
AU  - Mengyao Li
AU  - Jingrui Li
AU  - Yunfan Shi
AU  - Xiaomeng Zhang
AU  - Rui Liang
AU  - Feiya Xu
AU  - Siqi Ma
AU  - Jintao Chen
AU  - Yanshuang Lyu
AU  - Yuanhua Yang
AU  - Zhe Cheng
AU  - Yutao Guo
AU  - Lei Xia
AU  - Wei Wang
AU  - Bing Liu
AU  - Cunbo Jia
AU  - Shuai Zhang
AU  - Qian Gao
AU  - Yunxia Zhang
AU  - Zhu Zhang
AU  - Wanmu Xie
AU  - Guohui Fan
AU  - Shengfeng Wang
AU  - Zhenguo Zhai
PY  - 2026
DA  - 2026/07/09
JO  - NPJ Digital Medicine
DO  - 10.1038/s41746-026-02929-3
UR  - https://doi.org/10.1038/s41746-026-02929-3
AB  - Venous thromboembolism (VTE) is a common but preventable complication in hospitalized patients, yet standardized risk scoring using unstructured electronic health records (EHRs) remains challenging. We retrospectively analyzed anonymized EHRs from a nationwide multicenter cohort across 30 hospitals. Using stratified random sampling, 50 cases from 10 hospitals were used for development and 200 cases from the remaining 20 hospitals for testing. Expert knowledge-augmented prompts for the Padua and Caprini scores were developed through a Delphi process involving 19 specialists and compared with basic and complex prompts. Performance was evaluated using six open-source large language models (LLMs), with prevalence-adjusted bias-adjusted kappa (PABAK) and F1 score as primary metrics. Expert knowledge-augmented prompts showed the highest performance in both datasets. In the test dataset, mean item-level PABAK and F1 scores were 0.97 and 0.96 for Padua, and 0.97 and 0.93 for Caprini; most items showed PABAK and F1 scores above 0.90. Risk stratification was better for Padua (PABAK 0.92; F1 score 0.96) than for Caprini (PABAK 0.73; F1 score 0.64), with processing times of 6.07 s and 12.82 s per case, respectively. These findings indicated that LLMs with expert knowledge-augmented prompting could efficiently support automated Padua and Caprini scoring and risk stratification in thrombosis prevention workflows.
SN  - 2398-6352
ER  -