Using iterative agentic retrieval-augmented generation to refine the risk of infection from social contexts during the COVID-19 pandemic: Development and Evaluation Study
- 1KU Leuven. Department of Microbiology, Immunology and Transplantation. Laboratory of Clinical Microbiology.
Abstract
Background: Contact tracing is crucial to limit pathogen transmission and to gather epidemiological data. However, despite machine learning optimizations, it remains time-consuming. Large language models have been used for infection risk prediction, but no existing application converses with a user to adapt its risk estimates. Objective: We aimed to develop an iterative retrieval-augmented generation conversational framework to predict coronavirus disease 2019 (COVID-19) infection risk from textual social data. The conversational agent aims to gather personalized risk factors based on descriptions of social interactions. Methods: We trained a long short-term memory (LSTM) model to predict the risk of COVID-19 infection from 19,434 textual descriptions of social interactions, collected during the pandemic in the Leuven student community, in Belgium. We integrated this model into a Llama3.3-based conversational framework that iteratively gathers additional social information that may represent a risk of infection. We evaluated the framework using large language model (LLM)-as-a-judge metrics and by comparing its risk assessment in different social contexts. Results: Based solely on social interaction data, a fine-tuned RobBERT model predicted infection risk with an area under the receiver operating characteristic curve (ROC AUC) of 0.73. It performed similarly to an LSTM model and a logistic regression on term frequency-inverse document frequency (TF-IDF) features, which both achieved a ROC AUC of 0.72. These three models slightly outperformed a Quantized Low-Rank Adaptation (QLoRA)-fine-tuned Mistral NeMo model (ROC AUC = 0.69). Llama3.3 performed well in our framework: the predicted infection risk converged often, and it asked fewer redundant questions than the smaller models (Mistral 7B, Gemma3 12B). In addition, the framework showed the ability to stratify, by risk of infection, synthetic users who had different types of social interactions. Conclusions: Combining simple language models with large language models can refine individual infection risk from contact tracing data in a high-circulation context. Our conversational framework may help optimize contact tracing by highlighting social contexts with a high risk of infection.
Related
- Development and evaluation of Retrieval-Augmented Generation (RAG) and GraphRAG for guideline-based clinical decision support in chronic kidney disease
- Evaluation of retrieval-augmented generation (RAG) with GPT-4o for maxillofacial prosthodontics clinical questions.
- Applications, Technical Characteristics, and Evaluation Approaches of Retrieval-Augmented Generation in Nursing: Scoping Review
- Real-world evaluation of a multilingual agentic retrieval-augmented clinical AI framework for evidence-grounded medical decision support
- Retrieval-augmented generation in biomedicine: a survey of technologies, datasets, and clinical applications
BibTeX
@misc{thibaut2026using,
title = {{Using iterative agentic retrieval-augmented generation to refine the risk of infection from social contexts during the COVID-19 pandemic: Development and Evaluation Study}},
author = {J. Thibaut and M. Tunc and J. De Smedt and J. Raymenants and C. Geenen and L. Cavillot and E. Andre},
howpublished = {medRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.28.26366693},
url = {https://www.medrxiv.org/content/10.64898/2026.10.28.26366693v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Using iterative agentic retrieval-augmented generation to refine the risk of infection from social contexts during the COVID-19 pandemic: Development and Evaluation Study
AU - J. Thibaut
AU - M. Tunc
AU - J. De Smedt
AU - J. Raymenants
AU - C. Geenen
AU - L. Cavillot
AU - E. Andre
PY - 2026
DA - 2026/10/28
PB - medRxiv
DO - 10.64898/2026.10.28.26366693
UR - https://www.medrxiv.org/content/10.64898/2026.10.28.26366693v1
AB - Background: Contact tracing is crucial to limit pathogen transmission and to gather epidemiological data. However, despite machine learning optimizations, it remains time-consuming. Large language models have been used for infection risk prediction, but no existing application converses with a user to adapt its risk estimates. Objective: We aimed to develop an iterative retrieval-augmented generation conversational framework to predict coronavirus disease 2019 (COVID-19) infection risk from textual social data. The conversational agent aims to gather personalized risk factors based on descriptions of social interactions. Methods: We trained a long short-term memory (LSTM) model to predict the risk of COVID-19 infection from 19,434 textual descriptions of social interactions, collected during the pandemic in the Leuven student community, in Belgium. We integrated this model into a Llama3.3-based conversational framework that iteratively gathers additional social information that may represent a risk of infection. We evaluated the framework using large language model (LLM)-as-a-judge metrics and by comparing its risk assessment in different social contexts. Results: Based solely on social interaction data, a fine-tuned RobBERT model predicted infection risk with an area under the receiver operating characteristic curve (ROC AUC) of 0.73. It performed similarly to an LSTM model and a logistic regression on term frequency-inverse document frequency (TF-IDF) features, which both achieved a ROC AUC of 0.72. These three models slightly outperformed a Quantized Low-Rank Adaptation (QLoRA)-fine-tuned Mistral NeMo model (ROC AUC = 0.69). Llama3.3 performed well in our framework: the predicted infection risk converged often, and it asked fewer redundant questions than the smaller models (Mistral 7B, Gemma3 12B). In addition, the framework showed the ability to stratify, by risk of infection, synthetic users who had different types of social interactions. Conclusions: Combining simple language models with large language models can refine individual infection risk from contact tracing data in a high-circulation context. Our conversational framework may help optimize contact tracing by highlighting social contexts with a high risk of infection.
ER -