auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation
- 1Department of Psychology , Stanford University
- 2Department of Computer Science , Stanford University
Abstract
AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data. Data collection is a major bottleneck in this pipeline, however. Psychology, and computational cognitive science in particular, is well-positioned to benefit from AI experimentation because theories are often represented as code and crowdsourcing platforms enable programmatic human data collection at scale. Here, we apply automated discovery techniques to the project of generating theories in computational cognitive science, with an agent-based system collecting human data independently through crowdsourced survey experiments. As a testbed, we use a classic case study from cognitive psychology: judging which sequences of coin flips seem subjectively more random. Our system, auto-psych, uses nested agent-based discovery loops to generate explanatory theories of human behavior. The inner loop conjectures, fits, and critiques probabilistic cognitive models; the outer loop designs experiments to test these models, launches them online, and analyzes the data. This system can quickly and reliably recover ground-truth theories from synthetic data via systematic experimentation, but the nested structure is critical to model performance. Further, in three independent sequences of human experiments, the system finds theories that fit the data better than theories generated from the scientific literature. This work thus demonstrates the feasibility of automated data collection and theory discovery in computational cognitive science.
Related
- Limiter l’expérimentation animale n’est pas une recherche empêchée, mais une science libérée
- Novel Target Combinations in Lung Squamous Cell Carcinoma proposed by the Emet AI Research Environment and supported by discovery stage experimentation
- The training–discovery paradox: Evidence from citizen science
- Data-Driven Virus Discovery with Virushunter and Virusgatherer
- Targeting PRMT5 Across Cancers and AI-Driven Discovery of Candidate Compounds
BibTeX
@misc{prystawski2026autopsych,
title = {{auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation}},
author = {Ben Prystawski and Kushin Mukherjee and Daniel Wurgaft and Linas Nasvytis and M. Li and Noah D. Goodman and Michael C. Frank},
howpublished = {arXiv (Cornell University)},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2606.26460},
url = {https://arxiv.org/abs/2606.26460},
note = {CC BY},
}
RIS
TY - UNPB
TI - auto-psych: Automating the science of mind using agent-driven theory discovery and experimentation
AU - Ben Prystawski
AU - Kushin Mukherjee
AU - Daniel Wurgaft
AU - Linas Nasvytis
AU - M. Li
AU - Noah D. Goodman
AU - Michael C. Frank
PY - 2026
DA - 2026/06/24
PB - arXiv (Cornell University)
DO - 10.48550/arxiv.2606.26460
UR - https://arxiv.org/abs/2606.26460
AB - AI-based scientific automation is increasingly possible by using agents to generate hypotheses, design experiments, and analyze data. Data collection is a major bottleneck in this pipeline, however. Psychology, and computational cognitive science in particular, is well-positioned to benefit from AI experimentation because theories are often represented as code and crowdsourcing platforms enable programmatic human data collection at scale. Here, we apply automated discovery techniques to the project of generating theories in computational cognitive science, with an agent-based system collecting human data independently through crowdsourced survey experiments. As a testbed, we use a classic case study from cognitive psychology: judging which sequences of coin flips seem subjectively more random. Our system, auto-psych, uses nested agent-based discovery loops to generate explanatory theories of human behavior. The inner loop conjectures, fits, and critiques probabilistic cognitive models; the outer loop designs experiments to test these models, launches them online, and analyzes the data. This system can quickly and reliably recover ground-truth theories from synthetic data via systematic experimentation, but the nested structure is critical to model performance. Further, in three independent sequences of human experiments, the system finds theories that fit the data better than theories generated from the scientific literature. This work thus demonstrates the feasibility of automated data collection and theory discovery in computational cognitive science.
SN - 2331-8422
ER -