CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
Abstract
Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through which human participants actually engage with experiments. At the same time, AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination. We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences. Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. Across four multimodal models, each run ten times per task, we find that finishing an experiment and reproducing its signatures are different things. Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. CogArena provides the tasks, scoring code, and a public leaderboard for new submissions.
Related
- An Open-Source Standardized Pipeline for Equitable Observations of Interactive Behavioral Dynamics: Theory-driven Measurement, Analysis, and Masking
- Toward standardized behavioral analysis in IntelliCage experiments
- OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells
- Dynamic hypernode injection and graph attention fusion for multimodal sentiment analysis of behavioral signals
- Interactive effects of moisture level, lactic acid bacteria, and ensiling duration on fermentation and microbial communities in barnyard millet silage
BibTeX
@misc{fernandez2026cogarena,
title = {{CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments}},
author = {Kianté Fernandez and Caitlin Chen and Jialu Zhou and Bartek Sadowski and Anthony C. Miceli and Ian Krajbich},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36484},
eprint = {2610.36484},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36484},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments
AU - Kianté Fernandez
AU - Caitlin Chen
AU - Jialu Zhou
AU - Bartek Sadowski
AU - Anthony C. Miceli
AU - Ian Krajbich
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.36484
UR - https://arxiv.org/abs/2610.36484
AB - Claims about the cognitive capabilities of large language models are typically tested by translating behavioral paradigms into text prompts, bypassing the visual interfaces and time pressure through which human participants actually engage with experiments. At the same time, AI agents can now complete online behavioral tasks and produce human-like data, which raises concerns that online datasets in the social and behavioral sciences are increasingly open to contamination. We introduce CogArena, a benchmark of ten experimental paradigms from the social and behavioral sciences. Agents complete the experiments through a standard web browser, working with the same visual stimuli and response interfaces that online participants use. A three-level scoring procedure evaluates whether the agent completed the task, how accurately it performed, and whether its data show the behavioral signatures each paradigm reliably produces in people. Across four multimodal models, each run ten times per task, we find that finishing an experiment and reproducing its signatures are different things. Many sessions did not finish, and those that did reproduced some signatures reliably and others not at all. CogArena provides the tasks, scoring code, and a public leaderboard for new submissions.
ER -