Same answer, different reasons: AI-moderated interviews uncover topics and reasoning that surveys miss.
- 1Surgo Health
Abstract
Surveys can reach large samples but capture only pre-defined answers, with identical responses often concealing very different reasoning. We tested whether AI-moderated interviews capture this reasoning at scale. In a pre-registered, randomized, counterbalanced design, 1,953 US adults from a probability-based panel and an opt-in panel completed three AI-moderated interviews as well as a section with closed-ended and open-text survey items. Topics were extracted from transcripts inductively, without a predefined codebook, and validated by a confirmatory follow-up. Participants endorsed 94% of topics as fair descriptions of what they said. 21-51% of topics from the AI interview had no counterpart in the closed-ended battery, and 17-37 percentage points net coverage over open text answers. Among respondents who gave identical closed-ended responses, their reasoning diverged widely in valence and content. AI-moderated interviews surface reasons that closed-ended and open-text items miss, and show how reasoning differs among people who answer alike.
Related
- Explaining the reasons for induced abortion in Fasa County, Iran: a qualitative content analysis study
- Reasons for direct restoration failure from childhood to adolescence: A birth cohort study
- 16,648 reasons to live instead of dying by suicide: insights from a computer-assisted content analysis
- Leveraging Skin Mutagenesis to Uncover Genetic Cancer Susceptibilities
- Music performance anxiety and restrictive dietary behaviors: a moderated serial mediation model
BibTeX
@misc{anwylirvine2026same,
title = {{Same answer, different reasons: AI-moderated interviews uncover topics and reasoning that surveys miss.}},
author = {A. Anwyl-Irvine and M. Economides and T. Engelthaler and S. Sutermaster and H. Kemp and S. Sgaier and P. Smittenaar},
howpublished = {medRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.01.26367029},
url = {https://www.medrxiv.org/content/10.64898/2026.11.01.26367029v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Same answer, different reasons: AI-moderated interviews uncover topics and reasoning that surveys miss.
AU - A. Anwyl-Irvine
AU - M. Economides
AU - T. Engelthaler
AU - S. Sutermaster
AU - H. Kemp
AU - S. Sgaier
AU - P. Smittenaar
PY - 2026
DA - 2026/11/01
PB - medRxiv
DO - 10.64898/2026.11.01.26367029
UR - https://www.medrxiv.org/content/10.64898/2026.11.01.26367029v1
AB - Surveys can reach large samples but capture only pre-defined answers, with identical responses often concealing very different reasoning. We tested whether AI-moderated interviews capture this reasoning at scale. In a pre-registered, randomized, counterbalanced design, 1,953 US adults from a probability-based panel and an opt-in panel completed three AI-moderated interviews as well as a section with closed-ended and open-text survey items. Topics were extracted from transcripts inductively, without a predefined codebook, and validated by a confirmatory follow-up. Participants endorsed 94% of topics as fair descriptions of what they said. 21-51% of topics from the AI interview had no counterpart in the closed-ended battery, and 17-37 percentage points net coverage over open text answers. Among respondents who gave identical closed-ended responses, their reasoning diverged widely in valence and content. AI-moderated interviews surface reasons that closed-ended and open-text items miss, and show how reasoning differs among people who answer alike.
ER -