How AI agents search
5 affiliations
- 1Department of Information Engineering, University of Pisa, Pisa, Italy.
- 2Department of Computer Science, Georgetown University, Washington, District of Columbia, United States of America.
- 3Department of Biostatistics, Bioinformatics, and Biomathematics, Georgetown University Medical Center, Washington, District of Columbia, United States of America.
- 4Department of Computer, Control and Management Engineering, Sapienza University of Rome, Rome, Italy.
- 5School of Computing Science, University of Glasgow, Glasgow, United Kingdom.
Abstract
Large Language Models (LLMs) have evolved from relying primarily on knowledge stored in their parametric memory to using non-parametric memory mechanisms such as Retrieval-Augmented Generation (RAG) to provide and ground knowledge. Recently, LLMs have begun to perform retrieval autonomously within a reasoning loop, an approach referred to as Agentic RAG. This shift introduces new querying behaviours, with implications for the design of retrieval systems that have yet to be evaluated. In this work, we introduce a large-scale dataset (ASQ) capturing how Agentic RAG systems perform multi-step search and reasoning. ASQ contains about 1M agentic search traces collected from open-source Agentic RAG systems over 3 corpora, 5 agent variants, and 6 retrieval settings. Our analysis of ASQ shows that, within this controlled benchmark setting, a single search query can trigger long chains of retrieval calls (up to 186) and follow query reformulation dynamics different from those observed in historical human-search data. Our results also reveal substantial behavioural variation across agent and retrieval configurations: changes in re-ranking and retrieval depth do not consistently affect search workloads in terms of search-call volume and trace length across agents. To our knowledge, publicly available production logs do not expose the agent, dataset, and retrieval configuration associated with each trace. ASQ provides a reusable resource for further analysing agent search behaviour and supporting future information retrieval studies on agentic workloads under controlled settings.
Related
- Lipid Extract from Ulva ohnoi Algae: Chemical Characterization, Antibacterial Potential, and Cytotoxicity in the Search for New Agents Against MDR Infections.
- Evaluating Autonomous LLM Agents Across Molecular Prediction and Optimization Benchmarks
- Animals or plants? Evolutionary branching of sessile versus mobile cognitive agents in noisy environments
- Editorial: Exploring natural extracts as sustainable antibacterial agents against aquaculture pathogens
- Neurodevelopmental disorders in infants treated with anti-VEGF agents vs. laser photocoagulation or cryotherapy for ROP
BibTeX
@article{pezzuti2026agents,
title = {{How AI agents search}},
author = {Francesca Pezzuti and Ophir Frieder and Fabrizio Silvestri and Sean MacAvaney and Nicola Tonellotto},
journal = {PLOS One},
year = {2026},
month = oct,
doi = {10.1371/journal.pone.0360157},
url = {https://doi.org/10.1371/journal.pone.0360157},
note = {CC BY},
}
RIS
TY - JOUR
TI - How AI agents search
AU - Francesca Pezzuti
AU - Ophir Frieder
AU - Fabrizio Silvestri
AU - Sean MacAvaney
AU - Nicola Tonellotto
PY - 2026
DA - 2026/10/21
JO - PLOS One
DO - 10.1371/journal.pone.0360157
UR - https://doi.org/10.1371/journal.pone.0360157
AB - Large Language Models (LLMs) have evolved from relying primarily on knowledge stored in their parametric memory to using non-parametric memory mechanisms such as Retrieval-Augmented Generation (RAG) to provide and ground knowledge. Recently, LLMs have begun to perform retrieval autonomously within a reasoning loop, an approach referred to as Agentic RAG. This shift introduces new querying behaviours, with implications for the design of retrieval systems that have yet to be evaluated. In this work, we introduce a large-scale dataset (ASQ) capturing how Agentic RAG systems perform multi-step search and reasoning. ASQ contains about 1M agentic search traces collected from open-source Agentic RAG systems over 3 corpora, 5 agent variants, and 6 retrieval settings. Our analysis of ASQ shows that, within this controlled benchmark setting, a single search query can trigger long chains of retrieval calls (up to 186) and follow query reformulation dynamics different from those observed in historical human-search data. Our results also reveal substantial behavioural variation across agent and retrieval configurations: changes in re-ranking and retrieval depth do not consistently affect search workloads in terms of search-call volume and trace length across agents. To our knowledge, publicly available production logs do not expose the agent, dataset, and retrieval configuration associated with each trace. ASQ provides a reusable resource for further analysing agent search behaviour and supporting future information retrieval studies on agentic workloads under controlled settings.
SN - 1932-6203
ER -