Perceived face similarity reflects inverse-generative and naturalistic discriminative objectives
Abstract
The perceptual representations supporting our ability to recognize faces remain a computational mystery. Deep neural networks offer mechanistic hypotheses for human face perception, but theoretically distinct models often make indistinguishable representational predictions for randomly sampled faces. To expose diagnostic differences among these hypotheses, we compared six neural network models sharing an architecture but trained on distinct tasks, using face pairs optimized to elicit contrasting model predictions ("controversial" pairs) alongside randomly sampled pairs. We tested model predictions against face-dissimilarity judgments from 864 human participants across stimulus sets differing in realism and pose variation. Models prioritizing high-level, invariant structures (trained via inverse rendering, face identification, or object classification) most robustly matched human judgments. Furthermore, models trained on natural images typically outperformed synthetic-trained counterparts. Together, these findings suggest that human face perception is shaped by mechanisms that infer latent causes of facial appearance, discount nuisance variation, and are tuned by natural image statistics.
Related
- Perceived GenAI competency and university teachers’ intention to continue using generative artificial intelligence: Extending the technology acceptance model
- Beta bursts carry a graded signal of semantic updating during naturalistic narrative comprehension
- From Exposure to Experience: Measuring Gaze-Based Visual Experience with Faces During Naturalistic Viewing
- A modelling challenge to improve preparedness for highly pathogenicavian influenza epidemics: objectives, design and synthetic datageneration
- The development of navigation and attentional abilities shapes how children explore naturalistic environments
BibTeX
@misc{guo2026perceived,
title = {{Perceived face similarity reflects inverse-generative and naturalistic discriminative objectives}},
author = {Wenxuan Guo and Heiko H. Schütt and Kamila Maria Jozwik and Katherine R. Storrs and Nikolaus Kriegeskorte and Tal Golan},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.12619},
eprint = {2605.12619},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.12619},
}
RIS
TY - UNPB
TI - Perceived face similarity reflects inverse-generative and naturalistic discriminative objectives
AU - Wenxuan Guo
AU - Heiko H. Schütt
AU - Kamila Maria Jozwik
AU - Katherine R. Storrs
AU - Nikolaus Kriegeskorte
AU - Tal Golan
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2605.12619
UR - https://arxiv.org/abs/2605.12619
AB - The perceptual representations supporting our ability to recognize faces remain a computational mystery. Deep neural networks offer mechanistic hypotheses for human face perception, but theoretically distinct models often make indistinguishable representational predictions for randomly sampled faces. To expose diagnostic differences among these hypotheses, we compared six neural network models sharing an architecture but trained on distinct tasks, using face pairs optimized to elicit contrasting model predictions ("controversial" pairs) alongside randomly sampled pairs. We tested model predictions against face-dissimilarity judgments from 864 human participants across stimulus sets differing in realism and pose variation. Models prioritizing high-level, invariant structures (trained via inverse rendering, face identification, or object classification) most robustly matched human judgments. Furthermore, models trained on natural images typically outperformed synthetic-trained counterparts. Together, these findings suggest that human face perception is shaped by mechanisms that infer latent causes of facial appearance, discount nuisance variation, and are tuned by natural image statistics.
ER -