A compact generative language model learns the compositional grammar of phase-separating proteins
- 1Tata Institute of Fundamental Research Hyderabad Telangana India [email protected] [email protected].
- 2Department of Biomedical Engineering, McKelvey School of Engineering, Washington University in St. Louis Missouri 63130 USA [email protected].
- 3Department of Chemistry, Washington University in St. Louis Missouri 63130 USA.
Abstract
Liquid-liquid phase separation (LLPS) of intrinsically disordered proteins drives the formation of biomolecular condensates, yet the sequence-level grammar that promotes condensation remains difficult to specify from first principles. Existing protein language models are typically trained on large, folded-protein-rich sequence corpora and may not optimally capture the compositional biases of LLPS-prone disordered proteins. Here, we show that a compact generative language model, with an architecture and size (∼17 M parameters) selected via systematic hyperparameter optimization and cross-validation, trained from scratch on a small set of phase-separating protein sequences, can learn sequence features associated with condensate formation. The model generates thousands of sequences that reproduce length-dependent trends in disorder, charge decoration, hydropathy patterning, and sequence entropy, with training and generative behavior stable across independent seeds. Homology analysis indicates that generated sequences are not full-length copies of known proteins, but instead combine short segments matching natural proteins with a novel, model-generated sequence. Descriptor-based screening of short generated sequences yielded a pool of disordered candidates, from which five synthetically feasible peptides were selected for experimental testing. Four of them show condensate-like signatures under near-physiological conditions, including droplet-like assemblies and increased turbidity. Coarse-grained phase-coexistence simulations of a subset of generated sequences with longer lengths show a broad range of phase-separation tendencies, with 12 of 89 sequences comparable to the strongly phase-separating hnRNPA1 low-complexity domain. These results suggest that compact, task-specific generative models can learn information-rich sequence grammars from small curated datasets and propose experimentally testable condensate-forming peptides.
Related
- RNAGEN: A Generative Adversarial Network‐Based Model to Generate Synthetic RNA Sequences to Target Proteins
- Programmable design of functional proteins from natural language
- CEM 2.0: Co-evolution-guided mining of superior extremophilic proteins via a curated proteome database and protein language model.
- De novo design of ligand binding proteins using large language models alone
- Separating Detection Performance from Thermodynamic Interpretability in Quantum-Capacitance Affinity Assays.
BibTeX
@article{wasim2026compact,
title = {{A compact generative language model learns the compositional grammar of phase-separating proteins}},
author = {Abdul Wasim and Ushasi Pramanik and Anirban Das and Pikaso Latua and Jai S Rudra and Jagannath Mondal},
journal = {Chemical science},
year = {2026},
month = oct,
doi = {10.1039/d6sc03816f},
url = {https://europepmc.org/article/PMC/PMC13726131},
note = {CC BY-NC},
}
RIS
TY - JOUR
TI - A compact generative language model learns the compositional grammar of phase-separating proteins
AU - Abdul Wasim
AU - Ushasi Pramanik
AU - Anirban Das
AU - Pikaso Latua
AU - Jai S Rudra
AU - Jagannath Mondal
PY - 2026
DA - 2026/10/26
JO - Chemical science
DO - 10.1039/d6sc03816f
UR - https://europepmc.org/article/PMC/PMC13726131
AB - Liquid-liquid phase separation (LLPS) of intrinsically disordered proteins drives the formation of biomolecular condensates, yet the sequence-level grammar that promotes condensation remains difficult to specify from first principles. Existing protein language models are typically trained on large, folded-protein-rich sequence corpora and may not optimally capture the compositional biases of LLPS-prone disordered proteins. Here, we show that a compact generative language model, with an architecture and size (∼17 M parameters) selected via systematic hyperparameter optimization and cross-validation, trained from scratch on a small set of phase-separating protein sequences, can learn sequence features associated with condensate formation. The model generates thousands of sequences that reproduce length-dependent trends in disorder, charge decoration, hydropathy patterning, and sequence entropy, with training and generative behavior stable across independent seeds. Homology analysis indicates that generated sequences are not full-length copies of known proteins, but instead combine short segments matching natural proteins with a novel, model-generated sequence. Descriptor-based screening of short generated sequences yielded a pool of disordered candidates, from which five synthetically feasible peptides were selected for experimental testing. Four of them show condensate-like signatures under near-physiological conditions, including droplet-like assemblies and increased turbidity. Coarse-grained phase-coexistence simulations of a subset of generated sequences with longer lengths show a broad range of phase-separation tendencies, with 12 of 89 sequences comparable to the strongly phase-separating hnRNPA1 low-complexity domain. These results suggest that compact, task-specific generative models can learn information-rich sequence grammars from small curated datasets and propose experimentally testable condensate-forming peptides.
SN - 2041-6520
SN - 2041-6539
ER -