Leveraging Large Language Models to Improve Precision in Randomized Controlled Trials
Abstract
Large language models (LLMs) are increasingly used in statistical research and applications. However, they are also notorious for unreliable or biased information. Here, we explore whether LLMs can be used to improve the precision of randomized controlled trials (RCTs) in a safe and rigorous way. We consider two pathways for incorporating LLM assistance: (1) using LLMs to transform non-standard covariates, such as text, into numerical representations, and (2) leveraging the general knowledge embedded in LLMs to generate predicted outcomes. Although external data sources have been leveraged to improve precision before, the value added by LLM-generated predictions through these pathways remains an open question. We develop a pipeline for generally leveraging LLMs in this context and apply it to three different case studies. We find that we are able to safely improve precision, primarily through transforming non-standard covariates, but that LLMs also possess general knowledge that is useful in some contexts.
Related
- Probabilistic linkage with graphs: leveraging cluster structure to improve the accuracy of entity resolution at scale
- Informed forecasting: Leveraging auxiliary knowledge to boost large language model performance on time series forecasting
- Leakage-controlled Crowdfunding Success Prediction: Do Feature Selection and SMOTE Improve Boosting-Based Models?
- Model selection in linear mixed-effects analysis of craving outcomes in a randomized controlled trial of medications for opioid use disorders
- Variability among large language models in assessing CONSORT compliance of published randomized clinical trials
BibTeX
@misc{lowe2026leveraging,
title = {{Leveraging Large Language Models to Improve Precision in Randomized Controlled Trials}},
author = {Jaylin Lowe and Adam Sales and Johann A. Gagnon-Bartsch},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.30157},
eprint = {2605.30157},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.30157},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Leveraging Large Language Models to Improve Precision in Randomized Controlled Trials
AU - Jaylin Lowe
AU - Adam Sales
AU - Johann A. Gagnon-Bartsch
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2605.30157
UR - https://arxiv.org/abs/2605.30157
AB - Large language models (LLMs) are increasingly used in statistical research and applications. However, they are also notorious for unreliable or biased information. Here, we explore whether LLMs can be used to improve the precision of randomized controlled trials (RCTs) in a safe and rigorous way. We consider two pathways for incorporating LLM assistance: (1) using LLMs to transform non-standard covariates, such as text, into numerical representations, and (2) leveraging the general knowledge embedded in LLMs to generate predicted outcomes. Although external data sources have been leveraged to improve precision before, the value added by LLM-generated predictions through these pathways remains an open question. We develop a pipeline for generally leveraging LLMs in this context and apply it to three different case studies. We find that we are able to safely improve precision, primarily through transforming non-standard covariates, but that LLMs also possess general knowledge that is useful in some contexts.
ER -