Defining Applicability Domains for Metal–Ligand Selectivity Prediction: Predictive Uncertainty vs. Molecular Similarity
4 affiliations
- 1Kurnakov Institute of General and Inorganic Chemistry, Russian Academy of Sciences, Leninskii Prosp. 31, Moscow, Russian Federation, 119991
- 2A.N. Frumkin Institute of Physical Chemistry and Electrochemistry, Russian Academy of Sciences, Leninskii pr., 31, Building 4, Moscow, Russia, 119071
- 3Lomonosov Moscow State University, Leninskie Gory, Moscow, Russian Federation, 119991
- 4HSE University, Myasnitskaya Ulitsa, 20, Moscow, Russia, 101000
Abstract
Applicability domains are widely used to determine whether a machine-learning prediction can be trusted, yet they are commonly defined by similarity to the training data rather than by evidence that similarity predicts error. Here, we treat an applicability criterion as a testable claim and evaluate whether it identifies predictions with lower error than an equally sized random subset on a common held-out population. We test 23 criteria on a context-dependent metal–ligand selectivity problem, comparing molecular fingerprints, descriptor-space distances, learned embeddings, training support and predictive uncertainty. At 50% retention, predictive uncertainty reduces mean-of-pairs RMSE by 65%, compared with 32% for the best molecular distance and 18% for the best ligand-only distance. Fingerprint nearest-neighbour and learned-embedding distances provide no useful error ordering, whereas uncertainty remains informative after accounting for training support and predicted magnitude. In a prospective-style selection task, ranking ligands by their standardized predicted margin above a selectivity threshold increases single-ligand success from 0.639±0.035 under random selection to 0.831± 0.059. These results show that, at least for the context-dependent metal-ligand selectivity task studied here, structural similarity to the training data is not, by itself, evidence of predictive reliability in context-dependent molecular prediction. Applicability criteria can instead be treated as empirical hypotheses and tested directly against held-out error before being used to define reliable prediction regions or benchmark difficulty.
Related
- Applicability-Domain-Aware Evaluation of Machine-Learning NMR Chemical-Shift Prediction in Small drug Molecules: GT-NMR for 13C and CASCADE for 1H
- Spatial Applicability of Fictitious Point Feedback for Frequency Response Prediction in Unidirectional Carbon Fiber-Reinforced Polymer Plates
- What Random Splits Measure in Polymer Glass-Transition Prediction: Chemistry Holdouts, Applicability Domains, and Prior Integration
- Beyond the Ligand Applicability Domain: Generalization and Local Resolution in Structure-Aware Foundation Models for Affinity Prediction
- Graph attention framework for site-selectivity prediction in undirected C(sp3)–H borylation
BibTeX
@misc{zlobin2026defining,
title = {{Defining Applicability Domains for Metal–Ligand Selectivity Prediction: Predictive Uncertainty vs. Molecular Similarity}},
author = {Ivan Zlobin and Ivan Bespalov and Mark Schneider and Maksim Shandybo and Nikolai Rekut and Alexander Pavlov},
howpublished = {ChemRxiv},
year = {2026},
month = oct,
doi = {10.26434/chemrxiv.15010241/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15010241/v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - Defining Applicability Domains for Metal–Ligand Selectivity Prediction: Predictive Uncertainty vs. Molecular Similarity
AU - Ivan Zlobin
AU - Ivan Bespalov
AU - Mark Schneider
AU - Maksim Shandybo
AU - Nikolai Rekut
AU - Alexander Pavlov
PY - 2026
DA - 2026/10/11
PB - ChemRxiv
DO - 10.26434/chemrxiv.15010241/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15010241/v1
AB - Applicability domains are widely used to determine whether a machine-learning prediction can be trusted, yet they are commonly defined by similarity to the training data rather than by evidence that similarity predicts error. Here, we treat an applicability criterion as a testable claim and evaluate whether it identifies predictions with lower error than an equally sized random subset on a common held-out population. We test 23 criteria on a context-dependent metal–ligand selectivity problem, comparing molecular fingerprints, descriptor-space distances, learned embeddings, training support and predictive uncertainty. At 50% retention, predictive uncertainty reduces mean-of-pairs RMSE by 65%, compared with 32% for the best molecular distance and 18% for the best ligand-only distance. Fingerprint nearest-neighbour and learned-embedding distances provide no useful error ordering, whereas uncertainty remains informative after accounting for training support and predicted magnitude. In a prospective-style selection task, ranking ligands by their standardized predicted margin above a selectivity threshold increases single-ligand success from 0.639±0.035 under random selection to 0.831± 0.059. These results show that, at least for the context-dependent metal-ligand selectivity task studied here, structural similarity to the training data is not, by itself, evidence of predictive reliability in context-dependent molecular prediction. Applicability criteria can instead be treated as empirical hypotheses and tested directly against held-out error before being used to define reliable prediction regions or benchmark difficulty.
ER -