A structural antibody benchmark for leakage-aware deep-learning evaluation
- 1The Hebrew University of Jerusalem
Abstract
Deep-learning methods for antibody structure prediction, antibody-antigen interaction modelling and design are advancing rapidly. However, comparisons across studies remain difficult because training and test sets are often constructed independently, and a temporal cutoff alone does not prevent train-test leakage. We present SABLE (Structural Antibody Benchmark for deep-Learning Evaluation), a versioned structural antibody resource that couples a fixed training collection with a leakage-controlled held-out test set for reproducible machine-learning development and evaluation. SABLE combines 16,511 experimental training entries with 327 manually reviewed test entries and 3,274 high-confidence, patent-derived AlphaFold3 models spanning 476 antigens. Candidate test structures were selected after the AlphaFold3 temporal cutoff and filtered against the training set using antibody and antigen sequence similarity filters. Each test entry records its nearest training set neighbour, allowing users to quantify remaining relatedness and stratify performance by similarity. Versioned releases provide metadata, processed structures, CDR annotations, redundancy labels and model-confidence fields. A Python/PyTorch API and standardised benchmark metrics provide reproducible database access and evaluation code without requiring additional antibody-structure processing.
Related
- How Well Does Structural MRI Classify Autism? A Leakage-Free, Site-Aware and Externally Validated Machine-Learning Evaluation in Two ABIDE Cohorts
- Window leakage in clinical EEG machine learning: matched subject-independent evaluation across ADHD and Parkinson cohorts
- Reproducibility and Leakage-Controlled Evaluation of Machine-Learning Models for Chronic Kidney Disease Classification
- Antibody CDR-H3 loop flexibility: Insights from X-ray crystallography, structural bioinformatics, and the limits of current deep learning methods
- Benchmark validity in graph neural network scoring of metabolic reaction activity on Recon3D: detecting label leakage, memorized noise and input-invariant models
BibTeX
@misc{cohen2026structural,
title = {{A structural antibody benchmark for leakage-aware deep-learning evaluation}},
author = {T. Cohen and H. Bhattacharya and M. Ozery-Flato and D. Schneidman-Duhovny},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.06.749672},
url = {https://www.biorxiv.org/content/10.64898/2026.09.06.749672v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - A structural antibody benchmark for leakage-aware deep-learning evaluation
AU - T. Cohen
AU - H. Bhattacharya
AU - M. Ozery-Flato
AU - D. Schneidman-Duhovny
PY - 2026
DA - 2026/09/06
PB - bioRxiv
DO - 10.64898/2026.09.06.749672
UR - https://www.biorxiv.org/content/10.64898/2026.09.06.749672v1
AB - Deep-learning methods for antibody structure prediction, antibody-antigen interaction modelling and design are advancing rapidly. However, comparisons across studies remain difficult because training and test sets are often constructed independently, and a temporal cutoff alone does not prevent train-test leakage. We present SABLE (Structural Antibody Benchmark for deep-Learning Evaluation), a versioned structural antibody resource that couples a fixed training collection with a leakage-controlled held-out test set for reproducible machine-learning development and evaluation. SABLE combines 16,511 experimental training entries with 327 manually reviewed test entries and 3,274 high-confidence, patent-derived AlphaFold3 models spanning 476 antigens. Candidate test structures were selected after the AlphaFold3 temporal cutoff and filtered against the training set using antibody and antigen sequence similarity filters. Each test entry records its nearest training set neighbour, allowing users to quantify remaining relatedness and stratify performance by similarity. Versioned releases provide metadata, processed structures, CDR annotations, redundancy labels and model-confidence fields. A Python/PyTorch API and standardised benchmark metrics provide reproducible database access and evaluation code without requiring additional antibody-structure processing.
ER -