Diagnosing the Demographic Distributions of LLM-Based Synthetic Persona Data: Reference-Distribution Dependence and Post Hoc Adjustment
Abstract
We examine how strongly demographic-distribution diagnostics of LLM-based synthetic persona data depend on the official statistics chosen as the reference. We compared the sex × age-group × province joint distribution of the 1,000,000 Nemotron-Personas-Korea (NPK) records with Korean official statistics using total variation distance (TVD). Against the resident-registration population for April 2026, the time of use, the upper bound on the share discrepancy of any subgroup defined by the three variables was 1.81 percentage points, the margin of error of a survey with about 2,900 respondents. This value, however, depended on the reference date and population definition of the official statistics. The closest candidate examined, the 2024 census distribution for Korean nationals, gave 0.56 percentage points; no monthly resident-registration reference was closer. The reference distributions themselves also moved: between January 2025, when NPK's distance was smallest, and April 2026, the resident-registration distribution moved by more than twice that minimum distance, as did the census Korean-national distribution between 2024 and 2025. Raking reduced the range of the distance across reference months to about one fifth. Cell poststratification was applicable because none of the 204 cells was empty, and neither scheme produced extreme weights. Demographic alignment is necessary but not sufficient for response validity, and even this basic diagnostic depends on the choice of reference distribution. Providers should therefore disclose the source, reference date, population definition, and joint-cell allocation procedure of the official statistics used for demographic attributes, and users should repeat the diagnosis and adjustment against statistics current at the time of use.
Related
- Exhaustive adjustment-set selection and causal effects for distribution-valued outcomes, with inference under latent Gaussian spatio-temporal dependence
- Hyperedge-Triggered Hawkes Processes for Diagnosing Higher-Order Temporal Dependence in Event Streams
- Copula-Based Bivariate Kumaraswamy-Teissier Distributions: Modeling Temperature-Rainfall Dependence and Compound Extremes
- Exact bounds on the distribution function of isotropic log-concave distributions
- Convergence of stationary distributions for a class of zero-range processes and its application to a wealth distribution model with debt
BibTeX
@misc{song2026diagnosing,
title = {{Diagnosing the Demographic Distributions of LLM-Based Synthetic Persona Data: Reference-Distribution Dependence and Post Hoc Adjustment}},
author = {Eunjeong Song and Sehee Hong},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.28668},
eprint = {2608.28668},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.28668},
}
RIS
TY - UNPB
TI - Diagnosing the Demographic Distributions of LLM-Based Synthetic Persona Data: Reference-Distribution Dependence and Post Hoc Adjustment
AU - Eunjeong Song
AU - Sehee Hong
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2608.28668
UR - https://arxiv.org/abs/2608.28668
AB - We examine how strongly demographic-distribution diagnostics of LLM-based synthetic persona data depend on the official statistics chosen as the reference. We compared the sex × age-group × province joint distribution of the 1,000,000 Nemotron-Personas-Korea (NPK) records with Korean official statistics using total variation distance (TVD). Against the resident-registration population for April 2026, the time of use, the upper bound on the share discrepancy of any subgroup defined by the three variables was 1.81 percentage points, the margin of error of a survey with about 2,900 respondents. This value, however, depended on the reference date and population definition of the official statistics. The closest candidate examined, the 2024 census distribution for Korean nationals, gave 0.56 percentage points; no monthly resident-registration reference was closer. The reference distributions themselves also moved: between January 2025, when NPK's distance was smallest, and April 2026, the resident-registration distribution moved by more than twice that minimum distance, as did the census Korean-national distribution between 2024 and 2025. Raking reduced the range of the distance across reference months to about one fifth. Cell poststratification was applicable because none of the 204 cells was empty, and neither scheme produced extreme weights. Demographic alignment is necessary but not sufficient for response validity, and even this basic diagnostic depends on the choice of reference distribution. Providers should therefore disclose the source, reference date, population definition, and joint-cell allocation procedure of the official statistics used for demographic attributes, and users should repeat the diagnosis and adjustment against statistics current at the time of use.
ER -