RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling
Abstract
Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. Native 10K pretraining preserves strong reconstruction at 10,240 tokens and, in a controlled long-context benchmark, maintains strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct short-context extrapolation, but induces substantially greater distal representation diffusion. Frozen RNA-type evaluations show that RIBOSPAN learns state-of-the-art RNA representations, with a particularly clear advantage on long RNAs. Across downstream biological benchmarks, RIBOSPAN emerges as the strongest encoder-only RNA foundation model, achieving state-of-the-art performance in both full-transcript biological property prediction and zero-shot mutation-fitness modeling. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning, biological prediction, and full-transcript mRNA design.
Related
- BOTANIC-1: a series of long-context plant genomic foundation models in the agentic era
- A hierarchical memory architecture overcomes context limits in long-horizon multi-agent computational modeling
- DeepD enables efficient genome modeling and controllable long-context sequence generation across domains of life
- Harmonised benchmarking of foundation models for single-cell and spatial transcriptomics reveals context-dependent generalisation
- Coupling of oceanographic state to the dark proteome: a foundation for genome-informed marine productivity modeling
BibTeX
@misc{wang2026ribospan,
title = {{RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling}},
author = {Ziyuan Wang and Bohao Tang and Fei Zhang and Shuo Han and Pengfei Liu},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.22849},
eprint = {2608.22849},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.22849},
}
RIS
TY - UNPB
TI - RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling
AU - Ziyuan Wang
AU - Bohao Tang
AU - Fei Zhang
AU - Shuo Han
AU - Pengfei Liu
PY - 2026
DA - 2026/08/28
PB - arXiv
DO - 10.48550/arxiv.2608.22849
UR - https://arxiv.org/abs/2608.22849
AB - Full-length RNAs, particularly messenger RNAs, often exceed the context lengths used to pretrain existing RNA foundation models, limiting complete-transcript modeling at single-nucleotide resolution. We present RIBOSPAN, a 1.61-billion-parameter bidirectional RNA foundation model natively pretrained with context lengths up to 10,240 nt. RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs. Native 10K pretraining preserves strong reconstruction at 10,240 tokens and, in a controlled long-context benchmark, maintains strong contextual responsiveness and context-specific representation separation while keeping perturbation-induced changes highly localized. Inference-time YaRN scaling recovers much of the contextual organization lost by direct short-context extrapolation, but induces substantially greater distal representation diffusion. Frozen RNA-type evaluations show that RIBOSPAN learns state-of-the-art RNA representations, with a particularly clear advantage on long RNAs. Across downstream biological benchmarks, RIBOSPAN emerges as the strongest encoder-only RNA foundation model, achieving state-of-the-art performance in both full-transcript biological property prediction and zero-shot mutation-fitness modeling. Building on the same backbone, we develop a multidimensionally conditioned discrete-diffusion framework for full-length mRNA generation and redesign, including synonymous-codon diffusion for protein-preserving CDS optimization. Together, RIBOSPAN establishes a powerful long-context foundation for transferable RNA representation learning, biological prediction, and full-transcript mRNA design.
ER -