Steering sequence generation in protein language models through Iterative Lookback Monte Carlo Sampling

Abstract

Protein language models (pLMs) leverage large-scale evolutionary data to generate novel sequences, but steering generation toward desired physicochemical properties without sacrificing diversity remains a major challenge. Existing approaches often induce severe diversity loss or require computationally expensive retraining. We introduce Iterative Lookback Monte Carlo (ILMC), a training-free inference-time sampling strategy that interleaves autoregressive elongation with Metropolis–Hastings refinement to approximate sampling from a maximum-entropy target distribution balancing generative quality and steering objectives. We show theoretically that this target distribution is entropy-maximizing under fixed generative quality and steering constraints, and empirically that ILMC produces more diverse samples than standard autoregressive baselines at matched generative quality. Using simple steering potentials, ILMC improves desired molecular properties, yielding higher predicted melting temperatures than both unsteered autoregressive sampling and compute-matched rejection sampling at matched generative quality. ILMC naturally applies to classifier-guided steering, where it outperforms purely autoregressive guidance in diversity while maintaining comparable enrichment of target properties. We validate ILMC on family-specific pLMs and on the multi-family model ProGen3.

Few-shot regulatory sequence design with a genomic language model
B. Hie, Y. A. Hao, M. Liu +7
bioRxiv28 Oct 2026PreprintAll rights reservedFull textSystems & synthetic biologybioRxiv
Capsid-specialized protein language models reveal higher-order viral architecture from sequence
S. Liu, S. Xia, H. Wang
bioRxiv6 Sep 2026PreprintCC BY-NC-NDFull textMicrobiologybioRxiv
OmniTCR: a foundation model unifying T cell receptor recognition prediction and conditional sequence generation
F. Zeng, D. Feng, D. Song +5
bioRxiv10 Sep 2026PreprintCC BYFull textBioinformatics & computational biologybioRxiv
RiboBridge couples RNA backbone generation to sequence design through a learned interaction topology
Z. Ma, Y. Zheng
bioRxiv15 Oct 2026PreprintCC BYFull textBioinformatics & computational biologybioRxiv
Language-guided domain adaptation improves protein sequence rep-resentations for antimicrobial resistance analysis
R. Ye, L. Li, Z. Geng +1
bioRxiv4 Nov 2026PreprintCC BYFull textBioinformatics & computational biologybioRxiv

Cite

BibTeX RIS

BibTeX

@article{calvanese2026steering,
  title = {{Steering sequence generation in protein language models through Iterative Lookback Monte Carlo Sampling}},
  author = {Francesco Calvanese and Gianluca Lombardi and Martin Weigt and Jorge Fernandez-De-Cossio-Diaz},
  journal = {Machine Learning : Science and Technology},
  year = {2026},
  month = oct,
  doi = {10.1088/2632-2153/ae923a},
  url = {https://hal.science/hal-05839533v1},
  note = {CC BY 4.0},
}

RIS

TY  - JOUR
TI  - Steering sequence generation in protein language models through Iterative Lookback Monte Carlo Sampling
AU  - Francesco Calvanese
AU  - Gianluca Lombardi
AU  - Martin Weigt
AU  - Jorge Fernandez-De-Cossio-Diaz
PY  - 2026
DA  - 2026/10/26
JO  - Machine Learning : Science and Technology
DO  - 10.1088/2632-2153/ae923a
UR  - https://hal.science/hal-05839533v1
AB  - Protein language models (pLMs) leverage large-scale evolutionary data to generate novel sequences, but steering generation toward desired physicochemical properties without sacrificing diversity remains a major challenge. Existing approaches often induce severe diversity loss or require computationally expensive retraining. We introduce Iterative Lookback Monte Carlo (ILMC), a training-free inference-time sampling strategy that interleaves autoregressive elongation with Metropolis–Hastings refinement to approximate sampling from a maximum-entropy target distribution balancing generative quality and steering objectives. We show theoretically that this target distribution is entropy-maximizing under fixed generative quality and steering constraints, and empirically that ILMC produces more diverse samples than standard autoregressive baselines at matched generative quality. Using simple steering potentials, ILMC improves desired molecular properties, yielding higher predicted melting temperatures than both unsteered autoregressive sampling and compute-matched rejection sampling at matched generative quality. ILMC naturally applies to classifier-guided steering, where it outperforms purely autoregressive guidance in diversity while maintaining comparable enrichment of target properties. We validate ILMC on family-specific pLMs and on the multi-family model ProGen3.
SN  - 2632-2153
SN  - 2632-2153
ER  -

Abstract

Related

Cite