Towards a foundation model for raw nanopore signal through discrete tokenization
Abstract
Nanopore sequencing reads nucleic acids as changes in ionic current that carry information beyond the base sequence. Self-supervised pretrained models have advanced genomic sequence modeling and single-cell analysis, but remain underexplored for raw nanopore signals, which are continuous and lack a native token vocabulary. Here we present Brieno, an approach combining reconstruction-based tokenization through residual vector quantization with a Transformer for contextual representation learning. We pretrain the Transformer using masked-token prediction on raw nanopore signals from 118 million reads. With frozen or fine-tuned features, the model identifies species without requiring reference genomes. It complements Dorado by accurately classifying reads left unclassified in native-barcode demultiplexing, approaches Dorado's basecalling performance across six genomes, and exceeds Dorado's specialized 5-methylcytosine model as well as a dedicated 6-methyladenine caller. Brieno also supports accurate CpG methylation detection with limited labeled data. Across tasks, pretrained contextual representations improve downstream performance over non-pretrained baselines and generalize across sequencing runs. Together, these findings support self-supervised pretraining as a basis for transferable representations of raw nanopore signals.
Related
- BioMod: Biological Module Tokenization for Single-Cell Foundation Modeling
- Rethinking One Gene, One Token: Transcriptome-Aware Tokenization for Single-Cell Foundation Models
- tRNAZAP: Deep learning toolkit for nanopore tRNA signal analysis
- Nest weave pattern in weaverbirds: a sexual signal selected through sensory drive?
- REN-former prioritizes candidate regulators of kidney disease-state transitions through single-cell foundation modeling and human genetics
BibTeX
@misc{cheng2026towards,
title = {{Towards a foundation model for raw nanopore signal through discrete tokenization}},
author = {G. Cheng and E. Pitkänen},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.05.764536},
url = {https://www.biorxiv.org/content/10.64898/2026.11.05.764536v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - Towards a foundation model for raw nanopore signal through discrete tokenization
AU - G. Cheng
AU - E. Pitkänen
PY - 2026
DA - 2026/11/05
PB - bioRxiv
DO - 10.64898/2026.11.05.764536
UR - https://www.biorxiv.org/content/10.64898/2026.11.05.764536v1
AB - Nanopore sequencing reads nucleic acids as changes in ionic current that carry information beyond the base sequence. Self-supervised pretrained models have advanced genomic sequence modeling and single-cell analysis, but remain underexplored for raw nanopore signals, which are continuous and lack a native token vocabulary. Here we present Brieno, an approach combining reconstruction-based tokenization through residual vector quantization with a Transformer for contextual representation learning. We pretrain the Transformer using masked-token prediction on raw nanopore signals from 118 million reads. With frozen or fine-tuned features, the model identifies species without requiring reference genomes. It complements Dorado by accurately classifying reads left unclassified in native-barcode demultiplexing, approaches Dorado's basecalling performance across six genomes, and exceeds Dorado's specialized 5-methylcytosine model as well as a dedicated 6-methyladenine caller. Brieno also supports accurate CpG methylation detection with limited labeled data. Across tasks, pretrained contextual representations improve downstream performance over non-pretrained baselines and generalize across sequencing runs. Together, these findings support self-supervised pretraining as a basis for transferable representations of raw nanopore signals.
ER -