Multi-Peptide Prompting Enables In-Context Learning in Protein Language
- 1Cornell University
Abstract
Protein language models (PLMs) are trained primarily on individual protein sequences, yet many peptide-discovery problems require inference from only a small number of labeled examples. Here, we show that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification. We introduce multi-peptide example prompts (MPEPs), in which demonstration peptides are concatenated with glycine spacers and used as context for scoring query peptides by their prompted probability. We evaluate this approach across a synthetic pattern-completion task, secondary-structure classification, and MHC-II binder prediction using both encoder-only ESM-2 models and decoder-only ProGen2 models. Across tasks, performance improves with the number of peptide examples and with model scale, indicating that PLMs can extract shared sequence-level properties from prompted examples. We further introduce a difference score that contrasts positive-example and negative-example MPEPs, reducing compositional biases in raw PLM probabilities and substantially improving classification. On MHC-II binder prediction, MPEP-based classification with larger ESM-2 models matches or exceeds low-data classifiers trained on frozen ESM-2 embeddings, while requiring no training. These results reveal an unexpected in-context inference capability in single-sequence PLMs and establish MPEP conditioning as a lightweight strategy for low-data peptide classification.
Related
- Coarse composition suffices: tabular in-context learning for multi-activity antimicrobial peptide profiling
- Sequence Engineering at Non-motif Modulator Residues Yields a Peptide That Effectively Targets a Single PDZ Protein in a Disease-relevant Cellular Context
- Linker-Encoded Enzymatic Topology Control Enables Automated Peptide Stapling with Reusable Immobilized mTGase
- Targeted Gastrointestinal Delivery of mRNA Lipid Nanoparticles Enables Systemic Vaccination and Glucagon-like Peptide-1 Metabolic Therapy.
- Peptide-Directed Ligand Design Enables Chemical Disruption of the Nrf2/MafG Coiled-Coil Transcription Factor Complex.
BibTeX
@misc{almonte2026multipeptide,
title = {{Multi-Peptide Prompting Enables In-Context Learning in Protein Language}},
author = {J. Almonte and M. T. Vu and A. Ahn and E. H. Thiede},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.26.747074},
url = {https://www.biorxiv.org/content/10.64898/2026.08.26.747074v2},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Multi-Peptide Prompting Enables In-Context Learning in Protein Language
AU - J. Almonte
AU - M. T. Vu
AU - A. Ahn
AU - E. H. Thiede
PY - 2026
DA - 2026/08/26
PB - bioRxiv
DO - 10.64898/2026.08.26.747074
UR - https://www.biorxiv.org/content/10.64898/2026.08.26.747074v2
AB - Protein language models (PLMs) are trained primarily on individual protein sequences, yet many peptide-discovery problems require inference from only a small number of labeled examples. Here, we show that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification. We introduce multi-peptide example prompts (MPEPs), in which demonstration peptides are concatenated with glycine spacers and used as context for scoring query peptides by their prompted probability. We evaluate this approach across a synthetic pattern-completion task, secondary-structure classification, and MHC-II binder prediction using both encoder-only ESM-2 models and decoder-only ProGen2 models. Across tasks, performance improves with the number of peptide examples and with model scale, indicating that PLMs can extract shared sequence-level properties from prompted examples. We further introduce a difference score that contrasts positive-example and negative-example MPEPs, reducing compositional biases in raw PLM probabilities and substantially improving classification. On MHC-II binder prediction, MPEP-based classification with larger ESM-2 models matches or exceeds low-data classifiers trained on frozen ESM-2 embeddings, while requiring no training. These results reveal an unexpected in-context inference capability in single-sequence PLMs and establish MPEP conditioning as a lightweight strategy for low-data peptide classification.
ER -