Beyond Cultural Persona Bias: Response Geometry as a Measure for Language Models and People
Abstract
Researchers are increasingly prompting language models with cultural personas so that they answer questionnaires as diverse human groups would. However, studies often report that these persona responses do not predict the models' behavior. We argue that whether the persona is Western or non-Western, the model's answers carry a cultural preference, and that this preference masks the variables that could predict its behavior. We examine the prompted preference at two levels: aggregate Big Five scores and response geometry, the correlation matrix between dimensions within each respondent, represented as a symmetric positive-definite (SPD) matrix. In DeepSeek-V3 and GPT-4o, when the item order is fixed, the prompted preference appears at both levels. Randomizing item order reduces it in the response geometry, while leaving it in the Big Five scores: the Big Five still separate the two personas at $95−−97\%$, while response geometry falls to chance. In randomized administration, the response geometry still contains information about the respondent. In DeepSeek-V4-Flash, SPD features are associated with whether the same instance sorts Asian American public figures as ``foreign'' in a text IAT (p = .047), while the Big Five do not. In human studies conducted using randomized item order, the response geometry identifies people from test-retest sessions in $81$ -- 85%, against chance 0.4%, and predicts sex and, weakly, age. In addition, SPD features also predict complex human behavior (e.g., cumulative GPA, R2 = 0.28, permutation p = .003), where the Big Five do not.
Related
- Coarse geometry of metric measure spaces
- Measure-free Koopman-von Neumann dynamics and noncommutative Geometry
- Closed Response Calculus for SLE Weldings and Weil--Petersson Kähler Geometry
- Breaking Homogeneity: Diversifying Persona Sets for Creative LLM Outputs
- Augmented Hypothesis Testing with Persona-Based LLM Simulations
BibTeX
@misc{yuan2026beyond,
title = {{Beyond Cultural Persona Bias: Response Geometry as a Measure for Language Models and People}},
author = {Yuan Yuan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2607.02368},
eprint = {2607.02368},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.02368},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Beyond Cultural Persona Bias: Response Geometry as a Measure for Language Models and People
AU - Yuan Yuan
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2607.02368
UR - https://arxiv.org/abs/2607.02368
AB - Researchers are increasingly prompting language models with cultural personas so that they answer questionnaires as diverse human groups would. However, studies often report that these persona responses do not predict the models' behavior. We argue that whether the persona is Western or non-Western, the model's answers carry a cultural preference, and that this preference masks the variables that could predict its behavior. We examine the prompted preference at two levels: aggregate Big Five scores and response geometry, the correlation matrix between dimensions within each respondent, represented as a symmetric positive-definite (SPD) matrix. In DeepSeek-V3 and GPT-4o, when the item order is fixed, the prompted preference appears at both levels. Randomizing item order reduces it in the response geometry, while leaving it in the Big Five scores: the Big Five still separate the two personas at $95−−97\%$, while response geometry falls to chance. In randomized administration, the response geometry still contains information about the respondent. In DeepSeek-V4-Flash, SPD features are associated with whether the same instance sorts Asian American public figures as ``foreign'' in a text IAT (p = .047), while the Big Five do not. In human studies conducted using randomized item order, the response geometry identifies people from test-retest sessions in $81$ -- 85%, against chance 0.4%, and predicts sex and, weakly, age. In addition, SPD features also predict complex human behavior (e.g., cumulative GPA, R^2 = 0.28, permutation p = .003), where the Big Five do not.
ER -