SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
Abstract
Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e., generative modeling in a latent space. We hypothesize that this multi-stage training is not necessary to obtain performant co-design models and thus present SimpleDesign, an effective multi-modal protein design model trained directly in the data space. SimpleDesign leverages a single-stage end-to-end objective that combines discrete cross-entropy for sequences and a regression objective for structures. In order to effectively model the difference in sequence and structure modalities, we develop a Mixture-of-Transformer architecture that allows modality-specific processing while keeping global self-attention over both modalities. We train SimpleDesign on over 2M sequence-structure pairs achieving strong performance across co-design and unconditional sequence/structure generation benchmarks.
Related
- KAT-Net: a hyperbolic routing framework for joint lysine acylation site prediction with shared sequence–structure representations
- Joint protein, mRNA and DNA sequence design and optimization with nucleotide-level Potts models
- Representation learning for low-data protein fitness prediction using sequence–structure embeddings and graph neural networks
- CBAT-Net: a unified CNN–BiLSTM–MultiHead transformer architecture for joint Q3/Q8 protein secondary structure prediction
- GeoDSSOP: Geometry-aware prediction of protein backbone order parameters from sequence and structure
BibTeX
@misc{lu2026simpledesign,
title = {{SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign}},
author = {Jiarui Lu and Yuyang Wang and Yizhe Zhang and Jiatao Gu and Navdeep Jaitly and Joshua M. Susskind and Miguel Ángel Bautista},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.03377},
eprint = {2609.03377},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.03377},
}
RIS
TY - UNPB
TI - SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
AU - Jiarui Lu
AU - Yuyang Wang
AU - Yizhe Zhang
AU - Jiatao Gu
AU - Navdeep Jaitly
AU - Joshua M. Susskind
AU - Miguel Ángel Bautista
PY - 2026
DA - 2026/09/03
PB - arXiv
DO - 10.48550/arxiv.2609.03377
UR - https://arxiv.org/abs/2609.03377
AB - Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Developing generative models capable of understanding this intrinsically multi-modal relationship is crucial for fields like drug discovery and protein engineering. Existing models often rely on a multi-stage training process where autoencoders that tokenize data into latent representations are trained in a first stage. Secondly, a generative model is trained on the latent representation of the autoencoder(s), i.e., generative modeling in a latent space. We hypothesize that this multi-stage training is not necessary to obtain performant co-design models and thus present SimpleDesign, an effective multi-modal protein design model trained directly in the data space. SimpleDesign leverages a single-stage end-to-end objective that combines discrete cross-entropy for sequences and a regression objective for structures. In order to effectively model the difference in sequence and structure modalities, we develop a Mixture-of-Transformer architecture that allows modality-specific processing while keeping global self-attention over both modalities. We train SimpleDesign on over 2M sequence-structure pairs achieving strong performance across co-design and unconditional sequence/structure generation benchmarks.
ER -