Using large language models to facilitate literature review and data extraction for infectious disease models: COVID-19 as a test case
- 1Department of Global and Environmental Health, School of Global Public Health, New York University
Abstract
Infectious disease transmission models are governed by parameters informed by systematic review of epidemiological literature. Large language models (LLMs) could facilitate this, but the reliability of the results to inform models has not been tested.
We built an open-source, end-to-end pipeline to simulate LLM performance in a hypothetical scenario where they were available to inform COVID-19 models developed during the first four months of 2020. It screened articles and extracted the reproduction number, serial interval, and incubation period from full-text PDFs. We applied it to 2,067 PubMed/medRxiv records published 31 December 2019-30 April 2020 using four models (GPT-5-mini, GPT-5.4, Claude Opus 4.8 and Gemini 2.5 Pro) and evaluated it against full-corpus human screening and 50-article extraction gold standards. We combined models post hoc, pooled the extracted values into an infectious disease (SEIR) model, and ran sensitivity analyses to test approaches to improve extraction accuracy.
Screening sensitivity was 0.72-0.95 and specificity 0.92-0.99. For articles reporting few values, extraction F1 was 0.90-0.96 with precision 0.91-1.00; across all articles, including those with dozens of stratified estimates, recall fell to 0.38-0.78. No fabricated values observed; errors were misassignments of values filed under the wrong parameter, or borrowed values treated as the studys own. Incomplete extraction from dense articles was mainly due to prompting and output format, not model capability. Ensembling allowed recall-precision trade-offs, and correctness increased with model agreement, from about 30% at one vote to 92-95% at four. The pipeline processed the corpus in hours versus an estimated 130-265 person-hours of manual effort.
Our results show that current models can extract transmission parameters from unstructured literature accurately enough to inform outbreak modelling. The bottleneck lies in task specification, and careful prompt and output schema design are key to reducing misassignment errors. Human effort is best directed at workflow development and provenance validation.
Related
- Infectious disease communication for international students: A survey of association-accredited Japanese language schools in Japan
- Generative Language Models for Disease Treatment Recommendations: A Systematic Literature Review
- Infectious endocarditis caused by Elizabethkingia spp.: Two case reports and a literature review
- Bern Sleep-Wake Registry: temporal extraction of comorbidities from unstructured clinical text using a large language model
- Deploying Local Large Language Models for Automated Article Screening in Scientific Literature Reviews
BibTeX
@misc{yang2026using,
title = {{Using large language models to facilitate literature review and data extraction for infectious disease models: COVID-19 as a test case}},
author = {X. Yang and C. Y. Lee and B. J. Quilty and L. Zhang and Y. Mu and M. Jit},
howpublished = {medRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.06.26362399},
url = {https://www.medrxiv.org/content/10.64898/2026.09.06.26362399v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Using large language models to facilitate literature review and data extraction for infectious disease models: COVID-19 as a test case
AU - X. Yang
AU - C. Y. Lee
AU - B. J. Quilty
AU - L. Zhang
AU - Y. Mu
AU - M. Jit
PY - 2026
DA - 2026/09/06
PB - medRxiv
DO - 10.64898/2026.09.06.26362399
UR - https://www.medrxiv.org/content/10.64898/2026.09.06.26362399v1
AB - Infectious disease transmission models are governed by parameters informed by systematic review of epidemiological literature. Large language models (LLMs) could facilitate this, but the reliability of the results to inform models has not been tested. We built an open-source, end-to-end pipeline to simulate LLM performance in a hypothetical scenario where they were available to inform COVID-19 models developed during the first four months of 2020. It screened articles and extracted the reproduction number, serial interval, and incubation period from full-text PDFs. We applied it to 2,067 PubMed/medRxiv records published 31 December 2019-30 April 2020 using four models (GPT-5-mini, GPT-5.4, Claude Opus 4.8 and Gemini 2.5 Pro) and evaluated it against full-corpus human screening and 50-article extraction gold standards. We combined models post hoc, pooled the extracted values into an infectious disease (SEIR) model, and ran sensitivity analyses to test approaches to improve extraction accuracy. Screening sensitivity was 0.72-0.95 and specificity 0.92-0.99. For articles reporting few values, extraction F1 was 0.90-0.96 with precision 0.91-1.00; across all articles, including those with dozens of stratified estimates, recall fell to 0.38-0.78. No fabricated values observed; errors were misassignments of values filed under the wrong parameter, or borrowed values treated as the studys own. Incomplete extraction from dense articles was mainly due to prompting and output format, not model capability. Ensembling allowed recall-precision trade-offs, and correctness increased with model agreement, from about 30% at one vote to 92-95% at four. The pipeline processed the corpus in hours versus an estimated 130-265 person-hours of manual effort. Our results show that current models can extract transmission parameters from unstructured literature accurately enough to inform outbreak modelling. The bottleneck lies in task specification, and careful prompt and output schema design are key to reducing misassignment errors. Human effort is best directed at workflow development and provenance validation.
ER -