Conditional Generation And Inpainting Of Non-coding RNA Sequences With Masked Discrete Diffusion
- 1Karlsruhe Institute of Technology
Abstract
Designing functional non-coding RNA (ncRNA) is fundamental to synthetic biology and RNA therapeutics, yet generative modelling for ncRNA has received far less attention than protein design. We present RNA-MDLM, a framework that extends Masked Discrete Language Models (MDLM) to the conditional generation and inpainting of ncRNA. We make two additions: first, conditioning on RNA-type representations from a pretrained RNA language model, and second, a modified classifier-free guidance scheme (Mod-CFG) that interpolates among conditional, unconditional, and random-sequence probabilities for better control. We also introduce REPAINT GAMES, a benchmark of seven structured masking tasks to probe a model's performance on sequence patterns, structural motifs, and base-pairing per RNA type. Our model is trained on 4.6 million ncRNA sequences spanning six evaluable classes. It produces sequences whose composition and folding statistics closely match natural RNAs. Through extensive ablation studies, we find that the embedding-conditioned model achieves the best balance of structural fidelity, biological novelty, and inpainting accuracy, and its class label steers generation far more strongly than a plain label baseline. We further show that a model trained on a smaller, class-balanced subset can appear more realistic mainly by copying abundant natural sequences rather than learning their rules. We also benchmark against a masked-diffusion model and a family-specific VAE on ribozyme families, and find that a type-conditioned model like ours and a per-family model are solving different tasks, which must be accounted for in a fair comparison. We will release the code and the trained models.
Related
- Self-Supervised Discovery of Discrete Local States in Noisy Image Sequences
- Automatic Generation of Novel Highly-Functional Molecular Sequences
- RNAbpFlow: base pair-augmented SE(3) flow matching for conditional RNA 3D structure generation
- Reframing enzyme function prediction as conditional generation
- DiffuChannel: A Conditional Denoising Diffusion Model that Synthesises Single-Molecule Patch-Clamp Data with Exactly Known Idealisation (Ground Truth)
BibTeX
@misc{upadhyay2026conditional,
title = {{Conditional Generation And Inpainting Of Non-coding RNA Sequences With Masked Discrete Diffusion}},
author = {U. Upadhyay and C. Dai and J. Herold and K. Sato and A. Schug},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.17.752279},
url = {https://www.biorxiv.org/content/10.64898/2026.09.17.752279v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Conditional Generation And Inpainting Of Non-coding RNA Sequences With Masked Discrete Diffusion
AU - U. Upadhyay
AU - C. Dai
AU - J. Herold
AU - K. Sato
AU - A. Schug
PY - 2026
DA - 2026/09/17
PB - bioRxiv
DO - 10.64898/2026.09.17.752279
UR - https://www.biorxiv.org/content/10.64898/2026.09.17.752279v1
AB - Designing functional non-coding RNA (ncRNA) is fundamental to synthetic biology and RNA therapeutics, yet generative modelling for ncRNA has received far less attention than protein design. We present RNA-MDLM, a framework that extends Masked Discrete Language Models (MDLM) to the conditional generation and inpainting of ncRNA. We make two additions: first, conditioning on RNA-type representations from a pretrained RNA language model, and second, a modified classifier-free guidance scheme (Mod-CFG) that interpolates among conditional, unconditional, and random-sequence probabilities for better control. We also introduce REPAINT GAMES, a benchmark of seven structured masking tasks to probe a model's performance on sequence patterns, structural motifs, and base-pairing per RNA type. Our model is trained on 4.6 million ncRNA sequences spanning six evaluable classes. It produces sequences whose composition and folding statistics closely match natural RNAs. Through extensive ablation studies, we find that the embedding-conditioned model achieves the best balance of structural fidelity, biological novelty, and inpainting accuracy, and its class label steers generation far more strongly than a plain label baseline. We further show that a model trained on a smaller, class-balanced subset can appear more realistic mainly by copying abundant natural sequences rather than learning their rules. We also benchmark against a masked-diffusion model and a family-specific VAE on ribozyme families, and find that a type-conditioned model like ours and a per-family model are solving different tasks, which must be accounted for in a fair comparison. We will release the code and the trained models.
ER -