Noise Ceilings for Molecular Potency Benchmarks: Measuring Label Noise from Independent ChEMBL Documents
Abstract
The same compound measured against the same target in different laboratories often receives different potency values, which limits how well any model can be scored against such labels. For 17 potency tasks (target × Ki or IC50) from ChEMBL 37, we estimate this limit per task from 3,292 molecules reported in at least two source documents and express model scores as a fraction of ceiling f (0: predicting the training mean; 1: reaching the estimated noise floor). The floors range from 0.31 to 0.79 pChEMBL units, but for some tasks the replicates come from a few document pairs, and a bootstrap over document pairs then gives intervals several times wider than one over molecules. After removing molecules found in MoleculeNet and TDC, we benchmark ECFP-based gradient boosting, random forest and MLP models and frozen ChemBERTa-77M and MoLFormer-XL embeddings with a boosted-tree head under random, scaffold and temporal splits. ECFP with a random forest reaches a mean f of 0.90, 0.73 and 0.36 on the three splits (random: median 0.82, range 0.80–1.03 over the floor’s uncertainty). The frozen embeddings reach 0.69/0.66, 0.55/0.54 and 0.25/0.25 (ChemBERTa/MoLFormer) and are below ECFP with gradient boosting on every split (Wilcoxon over tasks, Holm-adjusted p < 0.05); on five tasks, a D-MPNN and a fine-tuned ChemBERTa do not close the gap on random and scaffold splits. On five task–split combinations ECFP models score below the estimated floor (f > 1), which shows that the floor is uncertain for these tasks; we discuss mis-estimation and learnable document-level offsets as explanations. We report single-test-set minimum detectable effects and a reporting protocol, and release code and data.
Related
- C-MAGE: Automated Extraction and Organization Engine for Molecular Representations in Chemical Documents
- Reliability of Short Molecular Dynamics Runs as Screening Labels: Noise Ceilings, Time Dependence and Adaptive Replicate Allocation
- Accuracy and Efficiency Benchmarks of Pretrained Machine Learning Potentials for Molecular Simulations
- Yield Prediction and Condition Recommendation on High-Throughput Experimentation Datasets: Encoding Effects and Noise Ceilings
- Measuring molecular singlet oxygen (1O2*) from atmospheric photosensitizers: Intercomparison of techniques, irradiation setups, data analysis and protocol recommendations
BibTeX
@misc{lukin2026noise,
title = {{Noise Ceilings for Molecular Potency Benchmarks: Measuring Label Noise from Independent ChEMBL Documents}},
author = {Ruslan Lukin},
howpublished = {ChemRxiv},
year = {2026},
month = oct,
doi = {10.26434/chemrxiv.15011172/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15011172/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Noise Ceilings for Molecular Potency Benchmarks: Measuring Label Noise from Independent ChEMBL Documents
AU - Ruslan Lukin
PY - 2026
DA - 2026/10/28
PB - ChemRxiv
DO - 10.26434/chemrxiv.15011172/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15011172/v1
AB - The same compound measured against the same target in different laboratories often receives different potency values, which limits how well any model can be scored against such labels. For 17 potency tasks (target × Ki or IC50) from ChEMBL 37, we estimate this limit per task from 3,292 molecules reported in at least two source documents and express model scores as a fraction of ceiling f (0: predicting the training mean; 1: reaching the estimated noise floor). The floors range from 0.31 to 0.79 pChEMBL units, but for some tasks the replicates come from a few document pairs, and a bootstrap over document pairs then gives intervals several times wider than one over molecules. After removing molecules found in MoleculeNet and TDC, we benchmark ECFP-based gradient boosting, random forest and MLP models and frozen ChemBERTa-77M and MoLFormer-XL embeddings with a boosted-tree head under random, scaffold and temporal splits. ECFP with a random forest reaches a mean f of 0.90, 0.73 and 0.36 on the three splits (random: median 0.82, range 0.80–1.03 over the floor’s uncertainty). The frozen embeddings reach 0.69/0.66, 0.55/0.54 and 0.25/0.25 (ChemBERTa/MoLFormer) and are below ECFP with gradient boosting on every split (Wilcoxon over tasks, Holm-adjusted p < 0.05); on five tasks, a D-MPNN and a fine-tuned ChemBERTa do not close the gap on random and scaffold splits. On five task–split combinations ECFP models score below the estimated floor (f > 1), which shows that the floor is uncertain for these tasks; we discuss mis-estimation and learnable document-level offsets as explanations. We report single-test-set minimum detectable effects and a reporting protocol, and release code and data.
ER -