LIMBER: A Hybrid RNA Foundation Model for Predicting and Understanding RNA Structure and Function
- 1University of Notre Dame
Abstract
RNA foundation models offer a promising route to decoding the relationships between RNA sequence, structure and function, yet how model shapes the acquisition and organization of biological information remains poorly understood. Here we introduce Layer-Interleaved Mamba-Bidirectional Encoder for RNA (LIMBER), a hybrid foundation model integrating bidirectional self-attention with selective state-space modeling, pretrained on 40.5 million non-coding RNA sequences. Through task-specific fine-tuning, LIMBER achieves leading performance on secondary-structure prediction and non-coding RNA classification benchmarks, while demonstrating strong transferability to translation-related tasks and RNA--small-molecule interaction prediction. Beyond model development, we establish a unified, task-agnostic framework to systematically evaluate and interpret the biological knowledge intrinsically acquired by RNA foundation models. This framework validates LIMBER's strong capabilities in recovering RNA structures and predicting mutational effects, while uncovering a hierarchical, task-dependent organization of different biological information across model layers. By further systematically training and evaluating a broad panel of model variants spanning model capacity, training duration and dataset size, we uncover distinct scaling behaviors across structural, evolutionary and functional capabilities. Together, these findings establish LIMBER as a versatile RNA foundation model and provide broader insights into how architectural design, internal representations and scaling shape the learning of RNA sequence-structure-function relationships.
Related
- Slide-level batch structure limits histology-guided supervision of transcriptomic foundation models
- Understanding Gene Expression-to-Image Learning: Spatial Structure and Information Fidelity
- Understanding cognitive function in post-COVID-19 condition with fatigue: an interdisciplinary multimodal investigation
- Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models
- Evaluating a multispectral miniaturised fluorometer with three excitation channels for predicting phytoplankton community structure indices from BGC-Argo float observations
BibTeX
@misc{wang2026limber,
title = {{LIMBER: A Hybrid RNA Foundation Model for Predicting and Understanding RNA Structure and Function}},
author = {Z. Wang and Y. Qiu and Y. He and I. Thompson and H. Wang and A. Porter and A. Li},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.28.762670},
url = {https://www.biorxiv.org/content/10.64898/2026.10.28.762670v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - LIMBER: A Hybrid RNA Foundation Model for Predicting and Understanding RNA Structure and Function
AU - Z. Wang
AU - Y. Qiu
AU - Y. He
AU - I. Thompson
AU - H. Wang
AU - A. Porter
AU - A. Li
PY - 2026
DA - 2026/10/28
PB - bioRxiv
DO - 10.64898/2026.10.28.762670
UR - https://www.biorxiv.org/content/10.64898/2026.10.28.762670v1
AB - RNA foundation models offer a promising route to decoding the relationships between RNA sequence, structure and function, yet how model shapes the acquisition and organization of biological information remains poorly understood. Here we introduce Layer-Interleaved Mamba-Bidirectional Encoder for RNA (LIMBER), a hybrid foundation model integrating bidirectional self-attention with selective state-space modeling, pretrained on 40.5 million non-coding RNA sequences. Through task-specific fine-tuning, LIMBER achieves leading performance on secondary-structure prediction and non-coding RNA classification benchmarks, while demonstrating strong transferability to translation-related tasks and RNA--small-molecule interaction prediction. Beyond model development, we establish a unified, task-agnostic framework to systematically evaluate and interpret the biological knowledge intrinsically acquired by RNA foundation models. This framework validates LIMBER's strong capabilities in recovering RNA structures and predicting mutational effects, while uncovering a hierarchical, task-dependent organization of different biological information across model layers. By further systematically training and evaluating a broad panel of model variants spanning model capacity, training duration and dataset size, we uncover distinct scaling behaviors across structural, evolutionary and functional capabilities. Together, these findings establish LIMBER as a versatile RNA foundation model and provide broader insights into how architectural design, internal representations and scaling shape the learning of RNA sequence-structure-function relationships.
ER -