URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment
- 1Insilico Medicine AI Limited , Masdar City , Abu Dhabi , UAE
- 2Independent researcher
- 3Insilico Medicine Canada Inc. , Montreal , Quebec , Canada
Abstract
Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and general-purpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.
Related
- Digitalizing Manufacturing Chemistry Targets: A Benchmark for Retrosynthesis Tools
- Beyond Chemistry Knowledge: Embedding Pedagogical Content Knowledge through an AI-Augmented Retrosynthesis Program for Chemistry Education
- vibe-queue: provenance-aware job execution for computational chemistry across workstations and clusters
- Polymer Informatics Atlas: Property-Specific Representations, Chemistry-Aware Generalization, and Reliability-Aware Machine Learning Across Five Polymer Properties
- MatUQ: a benchmark for uncertainty-aware out-of-distribution materials property prediction with graph neural networks
BibTeX
@misc{zagribelnyy2026ursa,
title = {{URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment}},
author = {Bogdan Zagribelnyy and Ivan Ilin and Nikita Bondarev and Anton Morgunov and Arkadii Lin and Maksim Kuznetsov and Rim Shayakhmetov and Vladimir Aladinskiy and Alex Aliper and Alex Zhavoronkov},
howpublished = {arXiv (Cornell University)},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2607.04688},
url = {https://arxiv.org/abs/2607.04688},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - URSA: Chemistry-Aware Benchmark for Utilitarian Retrosynthesis Assessment
AU - Bogdan Zagribelnyy
AU - Ivan Ilin
AU - Nikita Bondarev
AU - Anton Morgunov
AU - Arkadii Lin
AU - Maksim Kuznetsov
AU - Rim Shayakhmetov
AU - Vladimir Aladinskiy
AU - Alex Aliper
AU - Alex Zhavoronkov
PY - 2026
DA - 2026/06/23
PB - arXiv (Cornell University)
DO - 10.48550/arxiv.2607.04688
UR - https://arxiv.org/abs/2607.04688
AB - Synthesis planning aiming to find pathways of reactions for a target molecule is one of the most important and challenging tasks in drug discovery. Recent progress has produced both specialized deep-learning retrosynthesis systems and general-purpose large language models, but objective comparison remains difficult due to the lack of flexible, chemically interpretable benchmarking protocols. In the current study, we are introducing the URSA (Utilitarian RetroSynthesis Assessment) evaluation framework that provides the opportunity to benchmark the synthetic routes not only from a formal perspective, such as convergence to commercially available starting materials, but also from a chemical plausibility perspective, mimicking the way expert chemists evaluate the reactions and routes. The study covers a comprehensive evaluation of both conventional end-to-end retrosynthesis solutions and LLMs for the synthesis planning task on a set of novel, diverse target molecules with undisclosed synthetic routes, which represent realistic tasks in the daily drug design routine. We find that while LLMs can support high-level strategic planning, they currently underperform specialized retrosynthesis models in reliably solving synthesis planning tasks.
SN - 2331-8422
ER -