CaliPPer: quantifying, predicting and improving AI model performance for binding prediction
Abstract
Binding prediction models accelerate therapeutic antibody and TCR discovery, but their performance on new datasets is unpredictable, often leading to low discovery rates. Density-ratio methods (PAPE, M-CBPE) provide label-free performance estimation for binary classification, but their assumptions and aggregate-only outputs limit binding prediction on neoepitopes, antigen variants and chemical scaffolds. Here we present CaliPPer (Calibration and Prediction of Performance), a post-hoc framework pairing a multi-chain Sample-to-Domain Distance (S2DD) with distance-aware Bayesian recalibration, operating at three resolutions: generalisability score, aggregate performance prediction, and per-sample confidence. Across ten models, eight architectures and two immune-receptor domains, CaliPPer attains distance--performance correlations |r|=0.80−−0.92, predicts AUROC/AP/F1 with mean absolute errors 0.008−−0.070, and improves AUROC by up to +0.20 on unseen epitopes/variants. Applied retrospectively to five published TCR, BCR, MHC--peptide and small-molecule studies, CaliPPer raises true discovery rates in all five (e.g.\ 0/5 → 3/5 confirmed neoantigens), providing a triage layer between computational prediction and experimental validation.
Related
- Improving sport horse selection using machine learning: Early prediction of elite-level show jumping performance
- Quantifying Uncertainty in Brain Age Predictions via Conformal Prediction
- Quantifying performance inflation from hidden sample dependence in biomedical image classification benchmarks
- Benchmarking Uncertainty and Improving Risk Ranking in Multi-task Bioactivity Prediction
- TransBind2: Improving Transcription Factor-DNA Binding Prediction with Multimodal Data and Bidirectional Cross Attention
BibTeX
@misc{zheng2026calipper,
title = {{CaliPPer: quantifying, predicting and improving AI model performance for binding prediction}},
author = {Jian-Qing Zheng and Hantao Lou and Zinan Yin and Sam Farrar and Yuze Zhou and Elie Antoun and Xiangxi Wang and Xuetao Cao and Tao Dong},
howpublished = {arXiv},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2606.07258},
eprint = {2606.07258},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.07258},
}
RIS
TY - UNPB
TI - CaliPPer: quantifying, predicting and improving AI model performance for binding prediction
AU - Jian-Qing Zheng
AU - Hantao Lou
AU - Zinan Yin
AU - Sam Farrar
AU - Yuze Zhou
AU - Elie Antoun
AU - Xiangxi Wang
AU - Xuetao Cao
AU - Tao Dong
PY - 2026
DA - 2026/06/05
PB - arXiv
DO - 10.48550/arxiv.2606.07258
UR - https://arxiv.org/abs/2606.07258
AB - Binding prediction models accelerate therapeutic antibody and TCR discovery, but their performance on new datasets is unpredictable, often leading to low discovery rates. Density-ratio methods (PAPE, M-CBPE) provide label-free performance estimation for binary classification, but their assumptions and aggregate-only outputs limit binding prediction on neoepitopes, antigen variants and chemical scaffolds. Here we present CaliPPer (Calibration and Prediction of Performance), a post-hoc framework pairing a multi-chain Sample-to-Domain Distance (S2DD) with distance-aware Bayesian recalibration, operating at three resolutions: generalisability score, aggregate performance prediction, and per-sample confidence. Across ten models, eight architectures and two immune-receptor domains, CaliPPer attains distance--performance correlations |r|=0.80−−0.92, predicts AUROC/AP/F1 with mean absolute errors 0.008−−0.070, and improves AUROC by up to +0.20 on unseen epitopes/variants. Applied retrospectively to five published TCR, BCR, MHC--peptide and small-molecule studies, CaliPPer raises true discovery rates in all five (e.g.\ 0/5 → 3/5 confirmed neoantigens), providing a triage layer between computational prediction and experimental validation.
ER -