From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?
- 1AMD Silo AI, Helsinki, Finland
Abstract
Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We developed MAGI, an open modular agent that authors objectives, launches and monitors optimization, interprets structure activity relationships, and revises its strategy accordingly. MAGI generates molecules either directly through the LLM or by delegating to REINVENT4, with scoring services interchangeable behind a common contract. We tested it across nine retrospective lead-optimization campaigns from three pharmaceutical companies, replayed under fixed temporal cutoffs. Both routes produced valid structures: LLM proposals stayed closer to local chemistry and reached comparable or higher primary activity in fewer operations, whereas REINVENT explored broader chemical space. Whether a campaign met its objective depended on the predictive models, not on the generation route: attainment followed model accuracy on the chemistry proposed, dropping once that chemistry moved outside the model applicability domain. Separately, a blinded evaluation asked whether the MAGI output could pass as expert work: chemists were not able to discriminate agentic proposals from held-out compounds, and judged the SAR reasoning broadly plausible yet incomplete. Together, these results position MAGI as a coordination layer pluggable into existing computational chemistry workflows. The ceiling on real projects, however, remains currently set by scorer applicability rather than by tool orchestration.
Related
- CY-Bench: a comprehensive benchmark dataset for sub-national crop yield forecasting
- OmicsBench: a long-horizon multi-turn benchmark of AI agents on research-study-scale bioinformatics
- Discovery and Optimization of Imidazothiazole Carboxamides as Novel Anti-Tuberculosis Agents
- Integrated Approach for the Discovery of Antifungal and Antibiofilm Agents From Cerrado Plants
- Distributed AI agents enable comprehensive discovery of biological insights at scale
BibTeX
@misc{llompart2026from,
title = {{From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?}},
author = {P. Llompart and L. Guner and H. Lai and A. Tibo and Y. Xu},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.30.755603},
url = {https://www.biorxiv.org/content/10.64898/2026.09.30.755603v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - From Benchmark to Bench: Can Agents Survive Real-World Drug Discovery?
AU - P. Llompart
AU - L. Guner
AU - H. Lai
AU - A. Tibo
AU - Y. Xu
PY - 2026
DA - 2026/09/30
PB - bioRxiv
DO - 10.64898/2026.09.30.755603
UR - https://www.biorxiv.org/content/10.64898/2026.09.30.755603v1
AB - Agentic systems increasingly coordinate molecular-design tools, but it is unclear which layer of the stack limits outcomes on real projects. We developed MAGI, an open modular agent that authors objectives, launches and monitors optimization, interprets structure activity relationships, and revises its strategy accordingly. MAGI generates molecules either directly through the LLM or by delegating to REINVENT4, with scoring services interchangeable behind a common contract. We tested it across nine retrospective lead-optimization campaigns from three pharmaceutical companies, replayed under fixed temporal cutoffs. Both routes produced valid structures: LLM proposals stayed closer to local chemistry and reached comparable or higher primary activity in fewer operations, whereas REINVENT explored broader chemical space. Whether a campaign met its objective depended on the predictive models, not on the generation route: attainment followed model accuracy on the chemistry proposed, dropping once that chemistry moved outside the model applicability domain. Separately, a blinded evaluation asked whether the MAGI output could pass as expert work: chemists were not able to discriminate agentic proposals from held-out compounds, and judged the SAR reasoning broadly plausible yet incomplete. Together, these results position MAGI as a coordination layer pluggable into existing computational chemistry workflows. The ceiling on real projects, however, remains currently set by scorer applicability rather than by tool orchestration.
ER -