Learning Task-Specific Antibody Representations via Function-Aware Masking
Abstract
Antibody-specific language models pretrained via masked language modeling (MLM) learn representations that are critical for downstream sequence design and property prediction tasks. Yet, the corruption process itself is rarely leveraged as a source of inductive bias during pretraining. While preferentially masking complementarity-determining regions (CDRs) improves binding-related predictions, antibodies possess diverse biological priors over a variety of functions. Herein, we introduce function-aware masking, a family of pretraining algorithms that align mask placement with specific functional priors (e.g., from IMGT annotations or structure predictions) to shape the learned representation space. We show that these specialist masking strategies significantly improve performance on their respective objectives, yielding up to a 14% gain on structure-related tasks and up to a 5.9x improvement on CDR-related tasks. To further improve performance across multiple functional axes, we develop hybrid masking strategies that integrate multiple priors, balancing reconstruction over binding, structural, and biophysical objectives. Our results demonstrate that informed mask placement provides a parameter-free mechanism for imposing functional inductive biases in antibody language model training.
Related
- Base editing-mediated exon-specific knockout of glutamine synthetase generates a high-productivity CHO cell line for antibody manufacturing.
- Antibody CDR-H3 loop flexibility: Insights from X-ray crystallography, structural bioinformatics, and the limits of current deep learning methods
- iGC: Antibody Optimization via Reinforcement Learning over a Learned Somatic Hypermutation Prior
- Learning Topological Representations of Protein Structure and Dynamics
- AbLWR: a context-aware listwise ranking framework for antibody-antigen binding affinity prediction via positive-unlabeled learning
BibTeX
@misc{goel2026learning,
title = {{Learning Task-Specific Antibody Representations via Function-Aware Masking}},
author = {Ayan Goel and Thomas A. Walton and Amirali Aghazadeh},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.00518},
eprint = {2609.00518},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.00518},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning Task-Specific Antibody Representations via Function-Aware Masking
AU - Ayan Goel
AU - Thomas A. Walton
AU - Amirali Aghazadeh
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.00518
UR - https://arxiv.org/abs/2609.00518
AB - Antibody-specific language models pretrained via masked language modeling (MLM) learn representations that are critical for downstream sequence design and property prediction tasks. Yet, the corruption process itself is rarely leveraged as a source of inductive bias during pretraining. While preferentially masking complementarity-determining regions (CDRs) improves binding-related predictions, antibodies possess diverse biological priors over a variety of functions. Herein, we introduce function-aware masking, a family of pretraining algorithms that align mask placement with specific functional priors (e.g., from IMGT annotations or structure predictions) to shape the learned representation space. We show that these specialist masking strategies significantly improve performance on their respective objectives, yielding up to a 14% gain on structure-related tasks and up to a 5.9x improvement on CDR-related tasks. To further improve performance across multiple functional axes, we develop hybrid masking strategies that integrate multiple priors, balancing reconstruction over binding, structural, and biophysical objectives. Our results demonstrate that informed mask placement provides a parameter-free mechanism for imposing functional inductive biases in antibody language model training.
ER -