Building from the ground up: augmentation strategies to improve protein language model performance on protein sets
- 1University of Cincinnati
Abstract
An important underlying requirement for deep learning models is the availability of large datasets, however, in sciences such as chemistry and biology, generating experimental data of sufficient size represents a significant bottleneck. In this work, we probed the extent and nature of limited input data, characterize how several protein language models (pLMs) learned on suboptimal datasets, and investigated augmentation approaches to improve predictive model performance. We found that narrowing the distribution of protein-protein complexes in a limited dataset permits an order-of-magnitude decrease in training dataset size, if a model is only expected to generalize to in-distribution data. Similar results were found in active learning (AL), where both in- and out-of-distribution Matthews Correlation Coefficient (MCC) showed a small but consistent increase upon augmentation of the training set with complexes produced by ESM-3, a generative pLM.
Related
- Multiple strategies to improve the therapeutic efficacy of mesenchymal stem cell-derived exosome transplantation in ischemic stroke (Review)
- Data-driven strategies for enzyme engineering: From machine learning to protein language models
- AAV Gene Augmentation Therapy Rescues the Phenotype in the RDH5−/− Cat Model of Fundus Albipunctatus
- Low-pass filtering revisited: ramped phase randomization for cryo-EM map comparison and data augmentation
- Cytoplasmic Lipid Membrane Biogenesis in Prokaryotes: From Soluble Lipid Building Blocks to a Dynamic and Functional Boundary Layer
BibTeX
@misc{stump2026building,
title = {{Building from the ground up: augmentation strategies to improve protein language model performance on protein sets}},
author = {E. Stump and H. Dennison and P. Strobbia and G. Stan and R. I. Dima},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.20.760553},
url = {https://www.biorxiv.org/content/10.64898/2026.10.20.760553v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Building from the ground up: augmentation strategies to improve protein language model performance on protein sets
AU - E. Stump
AU - H. Dennison
AU - P. Strobbia
AU - G. Stan
AU - R. I. Dima
PY - 2026
DA - 2026/10/20
PB - bioRxiv
DO - 10.64898/2026.10.20.760553
UR - https://www.biorxiv.org/content/10.64898/2026.10.20.760553v1
AB - An important underlying requirement for deep learning models is the availability of large datasets, however, in sciences such as chemistry and biology, generating experimental data of sufficient size represents a significant bottleneck. In this work, we probed the extent and nature of limited input data, characterize how several protein language models (pLMs) learned on suboptimal datasets, and investigated augmentation approaches to improve predictive model performance. We found that narrowing the distribution of protein-protein complexes in a limited dataset permits an order-of-magnitude decrease in training dataset size, if a model is only expected to generalize to in-distribution data. Similar results were found in active learning (AL), where both in- and out-of-distribution Matthews Correlation Coefficient (MCC) showed a small but consistent increase upon augmentation of the training set with complexes produced by ESM-3, a generative pLM.
ER -