Retrieval-to-abstraction LLM-assisted literature synthesis: Case demonstrations in digital health technology-enabled randomized controlled trial outcome reporting

@article{harrison2026retrievaltoabstraction,
  title = {{Retrieval-to-abstraction LLM-assisted literature synthesis: Case demonstrations in digital health technology-enabled randomized controlled trial outcome reporting}},
  author = {Taylor B. Harrison and Dian Hu and Heling Jia and Shyang Hong Tan and Fang Chen and Zhongyu Zhang and Qiuhao Lu and Jinlian Wang and Liwei Wang and Ming Huang and Larry J. Prokop and Matt Hoy and Jennifer St. Sauver and Sunyang Fu and Hongfang Liu},
  journal = {PLOS Digital Health},
  year = {2026},
  month = oct,
  doi = {10.1371/journal.pdig.0001768},
  url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001768},
  note = {CC BY 4.0},
}

TY  - JOUR
TI  - Retrieval-to-abstraction LLM-assisted literature synthesis: Case demonstrations in digital health technology-enabled randomized controlled trial outcome reporting
AU  - Taylor B. Harrison
AU  - Dian Hu
AU  - Heling Jia
AU  - Shyang Hong Tan
AU  - Fang Chen
AU  - Zhongyu Zhang
AU  - Qiuhao Lu
AU  - Jinlian Wang
AU  - Liwei Wang
AU  - Ming Huang
AU  - Larry J. Prokop
AU  - Matt Hoy
AU  - Jennifer St. Sauver
AU  - Sunyang Fu
AU  - Hongfang Liu
PY  - 2026
DA  - 2026/10/09
JO  - PLOS Digital Health
DO  - 10.1371/journal.pdig.0001768
UR  - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001768
AB  - Systematic reviews of randomized controlled trials (RCTs) support evidence-based clinical advancement. Clinical trial reporting guidelines, including CONSORT, establish common data elements that support systematic synthesis of published trial outcomes. However, the rapid rise of digital health technologies presents new challenges for reporting guidelines, and substantial real-world reporting heterogeneity has been observed. Therefore, machine-assisted evaluation of reporting adherence offers a pathway for more agile information management to accommodate the diverse and disruptive nature of digital health technology-enabled RCTs. Here, we propose an accelerated retrieval-to-abstraction literature synthesis framework to conduct article screening, text and table data preprocessing, and full-text review using an unmodified pretrained large language model. We apply this methodology in two digital health technology-enabled RCT case studies evaluating adherence to technology and digital health equity guideline reporting items. Large language model performance was on par with inter-annotator agreement F1 scores (≥0.9) in both screening and information-extraction tasks. These findings support the scalability and rigor of accelerated evidence synthesis and demonstrate the importance of common reporting elements across digital health technology-enabled RCTs. Author summary: Evidence-based medicine aligns high quality scientific evidence with real-world healthcare by integrating research findings into clinical best practices. Typically, the gold standard for updating current practice with new evidence integration is conducted through the systematic review of multiple similar clinical trials. Aggregate level insights are helpful to ensure practice changes are guided by rigorous and consistent findings. In recent years, technology has been an area of increased interest both in trial evaluation and clinical adoption to improve both patient and provider experiences. However, the rapid and diverse nature of technology innovation presents challenges for systematic evidence synthesis of similar digital health technology-enabled trials. Large language models are a type of artificial intelligence particularly well suited for organization of large scale text data, which is the main task of a systematic review and reporting guideline adherence evaluation. Therefore, we designed a retrieval-to-abstraction literature synthesis framework to help us understand current technology and digital health equity reporting practices observed in published clinical trials. This work presents an important contribution because we found that a large language model preformed as well as human annotators, while being faster and more cost-effective at the information extraction task.
SN  - 2767-3170
ER  -