Disentangling Between-Target and Within-Target Signal in PPI Benchmarking
- 1Shanghai Jiao Tong University School of Medicine
Abstract
Pooled correlation, the metric usually reported when predicting mutation effects on protein–protein binding affinity (ΔΔG), mixes two different capabilities: placing targets on a common binding scale and ranking mutations correctly within a target. We formulate an exact between-/within-target covariance decomposition for PPI ΔΔG benchmarking, r = C_B + C_W, and apply it to matched, provenance-filtered evaluation sets. In a controlled comparison that changes only the split regime, on identical 7,523 mutation rows and identical model configurations, median pooled Pearson falls from 0.6844 under random splitting to 0.1508 under family-out-of-distribution evaluation, with ΔC_B = −0.4347 and ΔC_W = −0.1011; the between-target component accounts for 81.1% of the summed absolute component change. Target weighting changes how the components should be read: across five schemes C_B reverses sign in 40/109 configurations and in 34/109 even above |C_B| > 0.005, whereas C_W reverses in only 5/109. Fixed-K shortlist metrics are constrained by candidate-set geometry: a rank-linear discriminability surrogate is zero when n ≤ K and peaks near n* ≈ 20.5 for K = 10, and 40.8% of targets with n ≥ 5 fall in that degenerate regime. A matched retrospective ablation separated modifications that improve pooled scores only from those that also improve within-target ranking. External evidence from five methods is underpowered and its apparent cross-axis agreement is not robust to score orientation. Benchmarks should therefore report both components under declared weighting and candidate-set geometry.
Related
- Benchmarking methods for extracting microbial signal from host-dominated metatranscriptomes
- Characterise the Response of Amorphophallus konjac to Different Shading Gradients Through Non-Target Metabolomics
- An integrative multi-omics modeling framework for chemotherapy response prediction through microRNA–target regulatory signatures
- Brightness demixing for simultaneous multi-target imaging in 3D single-molecule localization microscopy
- From Predicted Ki to Surrogate IC50: Similarity-Guided Empirical Calibration of Drug Target Affinity Predictions
BibTeX
@misc{yu2026disentangling,
title = {{Disentangling Between-Target and Within-Target Signal in PPI Benchmarking}},
author = {Le Yu},
howpublished = {ChemRxiv},
year = {2026},
month = oct,
doi = {10.26434/chemrxiv.15010846/v1},
url = {https://chemrxiv.org/doi/full/10.26434/chemrxiv.15010846/v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Disentangling Between-Target and Within-Target Signal in PPI Benchmarking
AU - Le Yu
PY - 2026
DA - 2026/10/22
PB - ChemRxiv
DO - 10.26434/chemrxiv.15010846/v1
UR - https://chemrxiv.org/doi/full/10.26434/chemrxiv.15010846/v1
AB - Pooled correlation, the metric usually reported when predicting mutation effects on protein–protein binding affinity (ΔΔG), mixes two different capabilities: placing targets on a common binding scale and ranking mutations correctly within a target. We formulate an exact between-/within-target covariance decomposition for PPI ΔΔG benchmarking, r = C_B + C_W, and apply it to matched, provenance-filtered evaluation sets. In a controlled comparison that changes only the split regime, on identical 7,523 mutation rows and identical model configurations, median pooled Pearson falls from 0.6844 under random splitting to 0.1508 under family-out-of-distribution evaluation, with ΔC_B = −0.4347 and ΔC_W = −0.1011; the between-target component accounts for 81.1% of the summed absolute component change. Target weighting changes how the components should be read: across five schemes C_B reverses sign in 40/109 configurations and in 34/109 even above |C_B| > 0.005, whereas C_W reverses in only 5/109. Fixed-K shortlist metrics are constrained by candidate-set geometry: a rank-linear discriminability surrogate is zero when n ≤ K and peaks near n* ≈ 20.5 for K = 10, and 40.8% of targets with n ≥ 5 fall in that degenerate regime. A matched retrospective ablation separated modifications that improve pooled scores only from those that also improve within-target ranking. External evidence from five methods is underpowered and its apparent cross-axis agreement is not robust to score orientation. Benchmarks should therefore report both components under declared weighting and candidate-set geometry.
ER -