Skip to content
DraconiticResearch
Search Sources API
181,716 studies31 sourcesUpdated 7 min ago
Peer-reviewedCC BYProceedings of the National Academy of Sciences of the United States of America

Estimating the threat of AI-agent responding across online survey platforms.

  • Grace Zhang1
  • Robert Walatka2
  • Stephanie Y Chen3
  • Alexander K Moore4
  • Oleg Urminsky1
  • Kianté Fernandez5
  • Andrea Low6
  • Jonathan E Bogard7
  • Craig R Fox5,6
7 affiliations
  1. 1Marketing, University of Chicago Booth School of Business, Chicago, IL 60637.
  2. 2Behavioral Science, University of Chicago Booth School of Business, Chicago, IL 60637.
  3. 3Marketing, London Business School, London NW1 4SA, United Kingdom.
  4. 4Marketing, University of Illinois Chicago College of Business Administration, Chicago, IL 60607.
  5. 5Department of Psychology, University of California, Los Angeles, CA 90095.
  6. 6Behavioral Decision Making Area, Behavioral Decision Making Area, Anderson School of Management, University of California, Los Angeles, CA 90095.
  7. 7Organizational Behavior, Olin Business School, Organizational Behavior, Olin Business School, Washington University in St. Louis, St. Louis, MO 63130.
Read full textPDFDOICite

Abstract

Recent research and advances in LLMs have led to widespread concern that AI agents could pose as human online survey-takers. However, it remains unclear how prevalent AI agents are on survey research platforms and how to effectively detect AI agents. We validated a series of AI detection tests that effectively separated verified-human participants from both prompted commercial and purpose-built AI agents. Using these tests, we collected surveys on seven online platforms and find wide variation in rates of respondents failing AI tests, ranging from 6 to 41% across platforms (compared to a 2.4% false-positive rate for in-person human participants). We further demonstrate that undetected AI agents can impact the results of online surveys. Our findings suggest that while there is an urgent need for AI detection tests and consistent, systematic monitoring of data quality on online platforms, some platforms currently appear to provide data with a low rate of AI agents.

Related

  1. Beyond correlation: Intraclass correlation and bland-altman analysis reveal poor absolute agreement across multiplex cytokine immunoassay platforms
    Olfa Khalfallah, Cynthia Le Mouroux, Narjis Kraimi +9
    CytokinePeer-reviewedLicence not statedMolecular biology & gene regulationHAL
  2. Calibration, accuracy, and domain variation across one million collective forecasts from prediction platforms
    Justin Dickerson, Hilla Shinitzky, Maayan Peer +2
    Proceedings of the National Academy of Sciences of the United States of AmericaPeer-reviewedCC BY-NC-ND 4.0Full textMolecular biology & gene regulationPMC
  3. Benchmarking cell segmentation in two Spatialomics platforms with open-source software
    Mary E. Brown, Yuyu He, Erin Hudson +7
    Scientific ReportsPeer-reviewedCC BY-NC-ND 4.0Full textBiophysicsCrossref
  4. Hybrid exosome-liposome nanovesicles as delivery platforms for therapeutic applications: merging the best of both worlds.
    Jian Yu, Lujiao Yu, Fei Ren +3
    TheranosticsPeer-reviewedCC BYFull textMolecular biology & gene regulationEurope PMC
  5. scDRP: disentangled representation learning for predicting single-cell responses to perturbations and estimating individual treatment effects
    Briefings in bioinformaticsPeer-reviewedCC BYFull textMolecular biology & gene regulationEurope PMC

Cite

BibTeXRIS
BibTeX
@article{zhang2026estimating,
  title = {{Estimating the threat of AI-agent responding across online survey platforms.}},
  author = {Grace Zhang and Robert Walatka and Stephanie Y Chen and Alexander K Moore and Oleg Urminsky and Kianté Fernandez and Andrea Low and Jonathan E Bogard and Craig R Fox},
  journal = {Proceedings of the National Academy of Sciences of the United States of America},
  year = {2026},
  month = oct,
  doi = {10.1073/pnas.2606460123},
  url = {https://europepmc.org/article/MED/42879471},
  note = {CC BY},
}
RIS
TY  - JOUR
TI  - Estimating the threat of AI-agent responding across online survey platforms.
AU  - Grace Zhang
AU  - Robert Walatka
AU  - Stephanie Y Chen
AU  - Alexander K Moore
AU  - Oleg Urminsky
AU  - Kianté Fernandez
AU  - Andrea Low
AU  - Jonathan E Bogard
AU  - Craig R Fox
PY  - 2026
DA  - 2026/10/15
JO  - Proceedings of the National Academy of Sciences of the United States of America
DO  - 10.1073/pnas.2606460123
UR  - https://europepmc.org/article/MED/42879471
AB  - Recent research and advances in LLMs have led to widespread concern that AI agents could pose as human online survey-takers. However, it remains unclear how prevalent AI agents are on survey research platforms and how to effectively detect AI agents. We validated a series of AI detection tests that effectively separated verified-human participants from both prompted commercial and purpose-built AI agents. Using these tests, we collected surveys on seven online platforms and find wide variation in rates of respondents failing AI tests, ranging from 6 to 41% across platforms (compared to a 2.4% false-positive rate for in-person human participants). We further demonstrate that undetected AI agents can impact the results of online surveys. Our findings suggest that while there is an urgent need for AI detection tests and consistent, systematic monitoring of data quality on online platforms, some platforms currently appear to provide data with a low rate of AI agents.
SN  - 0027-8424
SN  - 1091-6490
ER  -