Benchmarking Uncertainty and Improving Risk Ranking in Multi-task Bioactivity Prediction
- 1Laboratory for Synthetic Chemistry and Chemical Biology Limited; State Key Laboratory of Synthetic Chemistry and Department of Chemistry, The University of Hong
Abstract
Multi-task bioactivity prediction transfers information across assays, but experimental prioritization also requires uncertainty estimates that identify unreliable predictions. We benchmarked prediction accuracy and uncertainty calibration for five multi-task predictors on 100 ChEMBL27 test assays and 100 CHEMBL37 assays under the random split and the clustering split. Calibration used native, Deep Ensemble and MC Dropout uncertainty; risk ranking was compared for two Gaussian process (GP) backbones. Adaptive deep kernel fitting (ADKF) achieved the best mean calibration results, while deep kernel transfer (DKT) ranked errors more effectively than ADKF with native uncertainty. We also introduce Influence Calibrated Support Reconstruction (ICSR), a risk score for kernel-based predictors. ICSR combines measured support reconstruction errors with query-specific influence and stabilizes their weighted average toward the full-support mean to adjust Gaussian process uncertainty while preserving predicted activities. With DKT and ADKF, ICSR improved all three mean risk-ranking metrics over native uncertainty and an adapted neighborhood comparator in every panel and split. ICSR achieved a mean half-query MAE reduction (R50) of 14.2-19.6%, where R50 measures the percentage decrease in mean absolute error (MAE) after retaining the lowest-risk half of the queries. A retrospectively selected SARS-CoV-2 main protease case further illustrated its use for selecting more reliable predictions. The benchmark supports joint assessment of calibration and error ranking, while ICSR improves selective use of kernel-based bioactivity predictions.
Related
- A literature-derived benchmarking database for single-ancestry polygenic risk score construction methods with spectral ranking inferences.
- An automated ranking platform for machine learning regression models for meat spoilage prediction using multi-spectral imaging and metabolic profiling
- GPCR Ligand Bioactivity Prediction with Physics-Informed Dual-State Query Learning
- A decision-based criterion and a resolution threshold for ranking genomic prediction models within trials
- Benchmarking Large Language Models for Assay-Aware Drug-Target Binding Affinity Prediction
BibTeX
@misc{wang2026benchmarking,
title = {{Benchmarking Uncertainty and Improving Risk Ranking in Multi-task Bioactivity Prediction}},
author = {Z. Wang and L. Tian and C.-M. Che},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.15.751772},
url = {https://www.biorxiv.org/content/10.64898/2026.09.15.751772v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Benchmarking Uncertainty and Improving Risk Ranking in Multi-task Bioactivity Prediction
AU - Z. Wang
AU - L. Tian
AU - C.-M. Che
PY - 2026
DA - 2026/09/15
PB - bioRxiv
DO - 10.64898/2026.09.15.751772
UR - https://www.biorxiv.org/content/10.64898/2026.09.15.751772v1
AB - Multi-task bioactivity prediction transfers information across assays, but experimental prioritization also requires uncertainty estimates that identify unreliable predictions. We benchmarked prediction accuracy and uncertainty calibration for five multi-task predictors on 100 ChEMBL27 test assays and 100 CHEMBL37 assays under the random split and the clustering split. Calibration used native, Deep Ensemble and MC Dropout uncertainty; risk ranking was compared for two Gaussian process (GP) backbones. Adaptive deep kernel fitting (ADKF) achieved the best mean calibration results, while deep kernel transfer (DKT) ranked errors more effectively than ADKF with native uncertainty. We also introduce Influence Calibrated Support Reconstruction (ICSR), a risk score for kernel-based predictors. ICSR combines measured support reconstruction errors with query-specific influence and stabilizes their weighted average toward the full-support mean to adjust Gaussian process uncertainty while preserving predicted activities. With DKT and ADKF, ICSR improved all three mean risk-ranking metrics over native uncertainty and an adapted neighborhood comparator in every panel and split. ICSR achieved a mean half-query MAE reduction (R50) of 14.2-19.6%, where R50 measures the percentage decrease in mean absolute error (MAE) after retaining the lowest-risk half of the queries. A retrospectively selected SARS-CoV-2 main protease case further illustrated its use for selecting more reliable predictions. The benchmark supports joint assessment of calibration and error ranking, while ICSR improves selective use of kernel-based bioactivity predictions.
ER -