What actually moves virtual screening performance: two measured resolution limits and a pre-registered decomposition of six interventions
- 1OceanSparx Pty Ltd
Abstract
Reported improvements to structure-based virtual screening are usually differences in AUROC or enrichment between two protocols on one benchmark. We asked what a benchmark of typical size can actually resolve, and then measured six interventions against that limit with the reading rule for each committed to version control before the data existed. Two limits, not one. Scoring the same Vina poses with a second implementation of the same scoring function—agreeing at r = 0.970, no compound differing by more than 2 kcal/mol—moves AUROC by 0.020 [95% CI 0.011, 0.029]. Re-running the whole protocol at a different search effort and scoring it the same way moves AUROC by 0.039 [0.024, 0.056]. The first bounds a comparison of scoring functions on fixed poses; the second bounds a comparison of protocols, which is what most published improvements are. Both fall inside the range in which docking improvements are typically reported. On SARS-CoV-2 Mpro: changing the scoring function on identical poses moved AUROC +0.140, repairing a receptor preparation defect +0.045, scoring the pose ensemble rather than the top pose +0.019, substituting DiffDock-L pose generation +0.017, supplying the correct binding site to a cofoldingmodel −0.013, and eight-fold search effort −0.019. Read against each arm’s own pre-registered bar, one is supported at full strictness, one directionally, and the rest are not. Read against the resolution limits measured here—which were fixed after four of the six bars, and so are reported alongside those readings rather than in place of them—one clears, one is marginal, and four fall inside. The search-effort result is the sharpest either way: its measured effect is smaller than the run-to-run reproducibility of the protocol it was measured in. Docking adds nothing demonstrable over seven free physicochemical descriptors on either panel we assembled: on Mpro it significantly subtracts (−0.002 [−0.004, −0.000]) and on Factor Xa its +0.015 [−0.003, +0.034] crosses zero. Applying the same descriptors to LIT-PCBA gives a median AUROC of 0.7260 across 15 targets against published AutoDock Vina at 0.581 and GNINA at 0.611–0.616. These descriptor baselines are supervised on each benchmark’s own labels while docking is not, so this is a bias control in the sense of Sieg et al., not a claim that descriptors are a deployable substitute. On the AVE-debiased variant the descriptor median falls to 0.6510. One method exceeded the floor. Boltz-2, an open-weights co-folding model scoring compounds from sequence and SMILES alone, does not beat the descriptor baseline on its own (Mpro +0.026, Factor Xa +0.019, both intervals spanning zero) but adds +0.093 [+0.062, +0.125] and +0.075 [+0.051, +0.099] respectively when combined with those descriptors. Its advantage is not that it is less property-driven—descriptors explain 63% of its score and 63% of Vina’s—but that on Mpro its residual, the part orthogonal to those descriptors, ranks actives at 0.642 where five docking scores we measured sat between 0.507 and 0.559.
Related
- Ultra-High-Performance Liquid Chromatography-High Resolution Mass Spectrometry and Gas Chromatography-Mass Spectrometry Screening of Classical Psychoactive Substances, Novel Psychoactive Substances, and Metabolites in Postmortem Urine Samples
- Synthesis of Atomically Precise Pt1Ag34 Bimetallic Nanoclusters and Their Catalytic Performance in the Thermal Decomposition of Ammonium Perchlorate
- High resolution mass spectrometry screening method for routine surveillance of pharmacologically active substance residues in meat and milk
- Large language model-based screening of substances and their composition from safety data sheets for high-resolution chemical exposure assessment
- Resolution-Robust Mid-IR ATR-FTIR Classification of Mixed Plastic Waste with PVC Screening and Polyethylene Subclassification
BibTeX
@misc{goodman2026what,
title = {{What actually moves virtual screening performance: two measured resolution limits and a pre-registered decomposition of six interventions}},
author = {John Goodman},
howpublished = {ChemRxiv},
year = {2026},
month = sep,
doi = {10.26434/chemrxiv.15008496/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008496/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - What actually moves virtual screening performance: two measured resolution limits and a pre-registered decomposition of six interventions
AU - John Goodman
PY - 2026
DA - 2026/09/08
PB - ChemRxiv
DO - 10.26434/chemrxiv.15008496/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15008496/v1
AB - Reported improvements to structure-based virtual screening are usually differences in AUROC or enrichment between two protocols on one benchmark. We asked what a benchmark of typical size can actually resolve, and then measured six interventions against that limit with the reading rule for each committed to version control before the data existed. Two limits, not one. Scoring the same Vina poses with a second implementation of the same scoring function—agreeing at r = 0.970, no compound differing by more than 2 kcal/mol—moves AUROC by 0.020 [95% CI 0.011, 0.029]. Re-running the whole protocol at a different search effort and scoring it the same way moves AUROC by 0.039 [0.024, 0.056]. The first bounds a comparison of scoring functions on fixed poses; the second bounds a comparison of protocols, which is what most published improvements are. Both fall inside the range in which docking improvements are typically reported. On SARS-CoV-2 Mpro: changing the scoring function on identical poses moved AUROC +0.140, repairing a receptor preparation defect +0.045, scoring the pose ensemble rather than the top pose +0.019, substituting DiffDock-L pose generation +0.017, supplying the correct binding site to a cofoldingmodel −0.013, and eight-fold search effort −0.019. Read against each arm’s own pre-registered bar, one is supported at full strictness, one directionally, and the rest are not. Read against the resolution limits measured here—which were fixed after four of the six bars, and so are reported alongside those readings rather than in place of them—one clears, one is marginal, and four fall inside. The search-effort result is the sharpest either way: its measured effect is smaller than the run-to-run reproducibility of the protocol it was measured in. Docking adds nothing demonstrable over seven free physicochemical descriptors on either panel we assembled: on Mpro it significantly subtracts (−0.002 [−0.004, −0.000]) and on Factor Xa its +0.015 [−0.003, +0.034] crosses zero. Applying the same descriptors to LIT-PCBA gives a median AUROC of 0.7260 across 15 targets against published AutoDock Vina at 0.581 and GNINA at 0.611–0.616. These descriptor baselines are supervised on each benchmark’s own labels while docking is not, so this is a bias control in the sense of Sieg et al., not a claim that descriptors are a deployable substitute. On the AVE-debiased variant the descriptor median falls to 0.6510. One method exceeded the floor. Boltz-2, an open-weights co-folding model scoring compounds from sequence and SMILES alone, does not beat the descriptor baseline on its own (Mpro +0.026, Factor Xa +0.019, both intervals spanning zero) but adds +0.093 [+0.062, +0.125] and +0.075 [+0.051, +0.099] respectively when combined with those descriptors. Its advantage is not that it is less property-driven—descriptors explain 63% of its score and 63% of Vina’s—but that on Mpro its residual, the part orthogonal to those descriptors, ranks actives at 0.642 where five docking scores we measured sat between 0.507 and 0.559.
ER -