Evaluating LLM-Generated Preference Distributions
Abstract
Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of the distributions they produce remain understudied. Here, we systematically analyze LLM-generated distributions of preferences for air travel, restaurants, and consumer products. Encouragingly, all models considered in our analysis exhibit self-coherence, with the most probable outcomes stabilizing rapidly under repeated sampling. At the same time, we observe substantial discordance across both model families and scales, with little consensus even among their most probable outcomes. These patterns hold across nine open-weight models, three choice domains, and show robustness under temperature changes, greedy decoding, and perturbations of prompt and ordering. Our findings indicate that outcomes are influenced more by the choice of model than by the wording of the prompt, challenging the common assumption that sufficiently capable LLMs produce similar preference distributions when used as stand-ins for survey respondents.
Related
- QTestBench: A Test Framework for Evaluating LLM-Generated Quantum Code
- Correlation Between Dopant Atom Evaporation Field and Measured Site Preference in Atom Probe Tomography
- Pronounced Site Preference in Cr-Doped Mn-Based M-Type Hexaferrites and Its Chemical Origins
- Photonically Generated Simultaneous Multi-Band Terahertz Communication System
- Analytical framework for computer-generated holography of screen space primitives with point-source integration in the Fourier domain
BibTeX
@misc{huang2026evaluating,
title = {{Evaluating LLM-Generated Preference Distributions}},
author = {Fan Huang and Minsuk Kim and C. Tyler Diggans and Filippo Radicchi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.01000},
eprint = {2610.01000},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.01000},
}
RIS
TY - UNPB
TI - Evaluating LLM-Generated Preference Distributions
AU - Fan Huang
AU - Minsuk Kim
AU - C. Tyler Diggans
AU - Filippo Radicchi
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.01000
UR - https://arxiv.org/abs/2610.01000
AB - Large Language Models (LLMs) are increasingly used as probabilistic generators for simulation, synthetic data generation, and decision support in settings where real-world data are unavailable. Yet, the structure and reliability of the distributions they produce remain understudied. Here, we systematically analyze LLM-generated distributions of preferences for air travel, restaurants, and consumer products. Encouragingly, all models considered in our analysis exhibit self-coherence, with the most probable outcomes stabilizing rapidly under repeated sampling. At the same time, we observe substantial discordance across both model families and scales, with little consensus even among their most probable outcomes. These patterns hold across nine open-weight models, three choice domains, and show robustness under temperature changes, greedy decoding, and perturbations of prompt and ordering. Our findings indicate that outcomes are influenced more by the choice of model than by the wording of the prompt, challenging the common assumption that sufficiently capable LLMs produce similar preference distributions when used as stand-ins for survey respondents.
ER -