Constructed Negatives Shape the Evaluation of Sequence-Based Protein–Protein Interaction Models
Abstract
Many machine-learning tasks, including representation learning, rely on negative examples that are constructed rather than directly observed. This issue is especially important for protein-protein interaction (PPI) prediction, where experimentally established non-interactions are scarce. PPI models are therefore commonly trained and evaluated against sampled, unobserved protein pairs treated as negatives. We study how this construction affects model comparison. Keeping positives, proteindisjoint splits, class balance, and model selection fixed, we vary training and evaluation negatives independently across six constructions and four sequence-based models, three of which use protein language model (pLM) representations. Changing only the test negatives shifts AUPRC by up to 0.26, and the top-ranked architecture changes with the construction. Negatives matched to true partners in pLM representation space reduce all pLM-based models to near-chance performance, and shuffling partners within each class shows that the original pairing contributes little to AUROC. Constructed training negatives rarely outperform uniform sampling: on external data, they underperform it in 108 of 120 comparisons for the pLM-based models, with gains never exceeding 0.05 AUROC and losses up to 0.23. We release splits, negative-generation procedures, and evaluation code that allow training and evaluation negatives to be varied separately. Our results show that with constructed negatives, model comparisons are not just noisy but systematically sensitive to how negatives are defined, a methodological choice that is often left implicit.
Related
- AIPFusion-Net: an interaction-fusion framework for anti-inflammatory peptide prediction using protein language model embeddings, local sequence patterns, and handcrafted descriptors.
- BioGraphBench-PPI enables audit-first evaluation of negative-sampling contracts and structural baselines in protein interaction networks
- ‘In vitro activity of cefepime-enmetazobactam on carbapenem-resistant gram negatives’—author's response
- Representation learning for low-data protein fitness prediction using sequence–structure embeddings and graph neural networks
- Sequence‐encoded dynamics diversify allostery within a conserved protein scaffold
BibTeX
@misc{durand2026constructed,
title = {{Constructed Negatives Shape the Evaluation of Sequence-Based Protein–Protein Interaction Models}},
author = {Enzo Durand and Sergei Grudinin and Béatrice Durand and Élodie Laine},
howpublished = {HAL},
year = {2026},
month = oct,
doi = {10.64898/2026.10.08.757593},
url = {https://hal.science/hal-05834555v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Constructed Negatives Shape the Evaluation of Sequence-Based Protein–Protein Interaction Models
AU - Enzo Durand
AU - Sergei Grudinin
AU - Béatrice Durand
AU - Élodie Laine
PY - 2026
DA - 2026/10/08
PB - HAL
DO - 10.64898/2026.10.08.757593
UR - https://hal.science/hal-05834555v1
AB - Many machine-learning tasks, including representation learning, rely on negative examples that are constructed rather than directly observed. This issue is especially important for protein-protein interaction (PPI) prediction, where experimentally established non-interactions are scarce. PPI models are therefore commonly trained and evaluated against sampled, unobserved protein pairs treated as negatives. We study how this construction affects model comparison. Keeping positives, proteindisjoint splits, class balance, and model selection fixed, we vary training and evaluation negatives independently across six constructions and four sequence-based models, three of which use protein language model (pLM) representations. Changing only the test negatives shifts AUPRC by up to 0.26, and the top-ranked architecture changes with the construction. Negatives matched to true partners in pLM representation space reduce all pLM-based models to near-chance performance, and shuffling partners within each class shows that the original pairing contributes little to AUROC. Constructed training negatives rarely outperform uniform sampling: on external data, they underperform it in 108 of 120 comparisons for the pLM-based models, with gains never exceeding 0.05 AUROC and losses up to 0.23. We release splits, negative-generation procedures, and evaluation code that allow training and evaluation negatives to be varied separately. Our results show that with constructed negatives, model comparisons are not just noisy but systematically sensitive to how negatives are defined, a methodological choice that is often left implicit.
ER -