When Can Digital Personas Reliably Approximate Human Survey Findings?
Abstract
Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains unclear when they can reliably approximate human survey findings. We answer this question using the LISS panel, constructing personas from respondents' background variables and pre-2023 survey histories, then testing them against the same respondents' held-out post-cutoff answers. Across four persona architectures, three LLMs, and two prediction tasks, we assess performance at the question, respondent, distributional, equity, and clustering levels. Digital personas improve alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure. Retrieval-augmented architectures provide the clearest gains, but performance depends more on human response structure than on model choice: personas perform best for low-variability questions and common respondent patterns, and worst for subjective, heterogeneous, or rare responses. Our results provide practical guidance on when digital personas could be appropriate for survey research and when human validation remains necessary.
Related
- The relationship between digital literacy and employment quality: Evidence from a Chinese Household Survey
- Efficiently and Reliably Measuring Information Processing Capacity in Dynamical Systems via Kernels
- Approximate Transitivity of Young Translation on Rough Paths
- Equivalence of Pseudo- and Approximate Amenability in Banach Algebras
- An Approximate-Diagonal Characterization of Amenable Actions on C*Algebras
BibTeX
@misc{jia2026when,
title = {{When Can Digital Personas Reliably Approximate Human Survey Findings?}},
author = {Mumin Jia and Yilin Chen and Divya Sharma and Jairo Diaz-Rodriguez},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.10659},
eprint = {2605.10659},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.10659},
}
RIS
TY - UNPB
TI - When Can Digital Personas Reliably Approximate Human Survey Findings?
AU - Mumin Jia
AU - Yilin Chen
AU - Divya Sharma
AU - Jairo Diaz-Rodriguez
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2605.10659
UR - https://arxiv.org/abs/2605.10659
AB - Digital personas powered by Large Language Models (LLMs) are increasingly proposed as substitutes for human survey respondents, yet it remains unclear when they can reliably approximate human survey findings. We answer this question using the LISS panel, constructing personas from respondents' background variables and pre-2023 survey histories, then testing them against the same respondents' held-out post-cutoff answers. Across four persona architectures, three LLMs, and two prediction tasks, we assess performance at the question, respondent, distributional, equity, and clustering levels. Digital personas improve alignment with human response distributions, especially in domains tied to stable attributes and values, but remain limited for individual prediction and fail to recover multivariate respondent structure. Retrieval-augmented architectures provide the clearest gains, but performance depends more on human response structure than on model choice: personas perform best for low-variability questions and common respondent patterns, and worst for subjective, heterogeneous, or rare responses. Our results provide practical guidance on when digital personas could be appropriate for survey research and when human validation remains necessary.
ER -