Are we measuring the right thing? Evaluation practice for machine-learned transition states and reaction barriers
- 1Department of Chemistry
- 2SP College
Abstract
Machine-learned models of chemical reactivity are now reported with structural deviations below 0.1 Å, barrier errors near 1 kcal/mol and search success rates above 90 %. These figures come from different tasks, reference quantities and denominators, and this review asks what they certify. Its evidence base is five molecular datasets of small-molecule organic saddle points computed with density-functional theory, one catalysis benchmark, and the models scored on them; enzymatic, condensed-phase, photochemical and solid-state barriers are outside its scope. We argue that a reported accuracy is a distance to a reference, summarised by one statistic, on one split, and that each of these choices can materially change apparent performance. A prior question is whether the electronic-energy barrier that is scored is the quantity the application uses. Five of the six datasets rely on proxy verification rather than an intrinsic reaction coordinate calculation at the final labelled level, while the sixth shows that exhaustive verification is feasible at scale. We then ask whether summary statistics carry the information a task consumes, whether one held-out set measures transfer along the relevant axis, and how the error of the reference compares with the thresholds used to rank methods. The kinetic and label-error results are existence proofs on named systems: they show that a mean error is not a universal certificate, not how often current models fail. We propose eight core reporting items with pass conditions and task applicability, a conditional grid in the supporting information, and four missing measurements, including a design for validating a sensitivity-weighted barrier error against downstream kinetic observables. No new electronicstructure, machine-learning or simulation calculation was performed.
Related
- Learning continuous reaction paths for transition-state prediction
- Transition states and dynamical bottlenecks in the Li-NC <-> Li-CN isomerizing reaction with the Lagrangian betweenness
- From Transition-State Geometry to Gap Times: What Lagrangian Betweenness Measures in Chemical Reaction Dynamics
- A Riemannian Framework for Chemical Reaction Dynamics: Geodesics, Curvature, and Transition-State Stability
- CatWalk: Reaction Pathway Generation and Transition State Sampling via Score-based Diffusion Models
BibTeX
@misc{sain2026measuring,
title = {{Are we measuring the right thing? Evaluation practice for machine-learned transition states and reaction barriers}},
author = {Dinesh Kumar Sain and DINESH KUMAR SAIN},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15009215/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009215/v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Are we measuring the right thing? Evaluation practice for machine-learned transition states and reaction barriers
AU - Dinesh Kumar Sain
AU - DINESH KUMAR SAIN
PY - 2026
DA - 2026/09/21
PB - ChemRxiv
DO - 10.26434/chemrxiv.15009215/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15009215/v1
AB - Machine-learned models of chemical reactivity are now reported with structural deviations below 0.1 Å, barrier errors near 1 kcal/mol and search success rates above 90 %. These figures come from different tasks, reference quantities and denominators, and this review asks what they certify. Its evidence base is five molecular datasets of small-molecule organic saddle points computed with density-functional theory, one catalysis benchmark, and the models scored on them; enzymatic, condensed-phase, photochemical and solid-state barriers are outside its scope. We argue that a reported accuracy is a distance to a reference, summarised by one statistic, on one split, and that each of these choices can materially change apparent performance. A prior question is whether the electronic-energy barrier that is scored is the quantity the application uses. Five of the six datasets rely on proxy verification rather than an intrinsic reaction coordinate calculation at the final labelled level, while the sixth shows that exhaustive verification is feasible at scale. We then ask whether summary statistics carry the information a task consumes, whether one held-out set measures transfer along the relevant axis, and how the error of the reference compares with the thresholds used to rank methods. The kinetic and label-error results are existence proofs on named systems: they show that a mean error is not a universal certificate, not how often current models fail. We propose eight core reporting items with pass conditions and task applicability, a conditional grid in the supporting information, and four missing measurements, including a design for validating a sensitivity-weighted barrier error against downstream kinetic observables. No new electronicstructure, machine-learning or simulation calculation was performed.
ER -