A self-supervised DNA foundation model with collapse-resistant multimodal fusion
- 1ModelsLive Inc.
Abstract
Genomic foundation models pretrained on DNA sequence have achieved strong performance across many tasks, but sequence-only representations cannot fully capture regulatory information from additional DNA-centric modalities. Existing multimodal genomic models are optimized for specific prediction tasks rather than reusable embeddings. Directly fusing heterogeneous modalities is challenging because sparse, peak-shaped regulatory signals and dense sequence embeddings have markedly different statistical structures, making naive alignment prone to near-zero solutions. We present a self-supervised DNA-centric multimodal foundation model integrating DNA sequence embeddings with local and global chromatin accessibility in a shared encoder to produce reusable window-level embeddings. We show that global normalization alleviates this collapse, enabling effective joint learning. The resulting embeddings improve regulatory activity prediction, regulatory signal ranking and chromatin accessibility peak detection, achieving a 4.6-fold AUPRC improvement over the DNA-only baseline, with further gains on external ClinVar, GTEx eQTL and PBMC caQTL datasets.
Related
- Metabolite-centric Knowledge Graph Unveils Predictive and Mechanistic Insights through Self-supervised Multimodal Learning
- Supervised 3D Virtual Fluorescence and Pseudo-H&E Staining of Fresh Bulk Tissues via Multimodal qOBM and Fluorescence Microscopy
- Learning from tandem mass spectra at scale with a self-supervised foundation model for proteomics
- MtReNIRS: A Self-Supervised Foundation Model for Transferable Neurovascular Representations
- HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning
BibTeX
@misc{chen2026selfsupervised,
title = {{A self-supervised DNA foundation model with collapse-resistant multimodal fusion}},
author = {Y. Chen},
howpublished = {bioRxiv},
year = {2026},
month = aug,
doi = {10.64898/2026.08.19.745697},
url = {https://www.biorxiv.org/content/10.64898/2026.08.19.745697v2},
note = {CC BY},
}
RIS
TY - UNPB
TI - A self-supervised DNA foundation model with collapse-resistant multimodal fusion
AU - Y. Chen
PY - 2026
DA - 2026/08/19
PB - bioRxiv
DO - 10.64898/2026.08.19.745697
UR - https://www.biorxiv.org/content/10.64898/2026.08.19.745697v2
AB - Genomic foundation models pretrained on DNA sequence have achieved strong performance across many tasks, but sequence-only representations cannot fully capture regulatory information from additional DNA-centric modalities. Existing multimodal genomic models are optimized for specific prediction tasks rather than reusable embeddings. Directly fusing heterogeneous modalities is challenging because sparse, peak-shaped regulatory signals and dense sequence embeddings have markedly different statistical structures, making naive alignment prone to near-zero solutions. We present a self-supervised DNA-centric multimodal foundation model integrating DNA sequence embeddings with local and global chromatin accessibility in a shared encoder to produce reusable window-level embeddings. We show that global normalization alleviates this collapse, enabling effective joint learning. The resulting embeddings improve regulatory activity prediction, regulatory signal ranking and chromatin accessibility peak detection, achieving a 4.6-fold AUPRC improvement over the DNA-only baseline, with further gains on external ClinVar, GTEx eQTL and PBMC caQTL datasets.
ER -