Imputation of Public Opinion Data with Large Language Models
Abstract
Large language models (LLMs) are widely evaluated as simulators of survey responses, yet applied survey research mostly faces item nonresponse. Imputation fills in those missing values to restore the overall structure of a partially observed dataset and has its own well-defined evaluation criteria, which differ fundamentally from prediction accuracy. We present the first evaluation of LLM imputation against proper multiple-imputation metrics and statistical baselines on more than 5 million imputed responses from four public opinion surveys under various missingness mechanisms. We evaluate in-context learning (ICL) for this task by investigating 84 ICL configurations and comparing the best-performing configurations on six open-weight LLMs. The strongest ICL approach improves on zero-shot persona prompting but remains, at best, competitive with statistical baselines (MICE PMM, TabPFN). ICL approaches are sensitive to survey topics and consume 2-3 orders of magnitude more compute. Moreover, prediction metrics commonly used in LLM survey simulation (macro-F1, TVD) do not reliably track imputation quality. Our findings highlight the importance of fit-for-purpose evaluation for LLM imputation, where established multiple imputation remains the better default.
Related
- Bounding MrP: Subnational public opinion estimation without detailed census data
- A Distributional Framework for Behavioral Characterization and Risk Management of Large Language Models
- Sequential Functional Structured Tucker Compression for Large Language Model Attentions
- Large-Scale Comparison of Metaheuristic Algorithms via Behavioral Verification of Large Language Model-Generated Codes
- The Probabilistic Structure of Large Language Models
BibTeX
@misc{holtdirk2026imputation,
title = {{Imputation of Public Opinion Data with Large Language Models}},
author = {Tobias Holtdirk and Georg Ahnert and Joseph W Sakshaug and Anna-Carolina Haensch},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.09351},
eprint = {2606.09351},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.09351},
}
RIS
TY - UNPB
TI - Imputation of Public Opinion Data with Large Language Models
AU - Tobias Holtdirk
AU - Georg Ahnert
AU - Joseph W Sakshaug
AU - Anna-Carolina Haensch
PY - 2026
DA - 2026/10/30
PB - arXiv
DO - 10.48550/arxiv.2606.09351
UR - https://arxiv.org/abs/2606.09351
AB - Large language models (LLMs) are widely evaluated as simulators of survey responses, yet applied survey research mostly faces item nonresponse. Imputation fills in those missing values to restore the overall structure of a partially observed dataset and has its own well-defined evaluation criteria, which differ fundamentally from prediction accuracy. We present the first evaluation of LLM imputation against proper multiple-imputation metrics and statistical baselines on more than 5 million imputed responses from four public opinion surveys under various missingness mechanisms. We evaluate in-context learning (ICL) for this task by investigating 84 ICL configurations and comparing the best-performing configurations on six open-weight LLMs. The strongest ICL approach improves on zero-shot persona prompting but remains, at best, competitive with statistical baselines (MICE PMM, TabPFN). ICL approaches are sensitive to survey topics and consume 2-3 orders of magnitude more compute. Moreover, prediction metrics commonly used in LLM survey simulation (macro-F1, TVD) do not reliably track imputation quality. Our findings highlight the importance of fit-for-purpose evaluation for LLM imputation, where established multiple imputation remains the better default.
ER -