BioMod: Biological Module Tokenization for Single-Cell Foundation Modeling
- 1Shenzhen Loop Area Instutite
Abstract
Single-cell foundation models typically represent cells as flat sequences of individual genes, requiring higher-order biological organization to be inferred implicitly from sparse expression profiles. Similar to the evolution from character-level to semantic tokenization in natural language processing,we hypothesize that cellular states can be more efficiently represented using biologically meaningful intermediate units. Here we present BioMod, a 50-million-parameter single-cell foundation model that introduces biological modules as representation units between genes and cells. BioMod constructs an overlapping vocabulary of 660 biological modules derived from curated pathways and data-driven gene programs, replacing thousands of gene-level inputs with compact module representations. Pretrained on approximately 73 million human cells using high-ratio module masking with an asymmetric decoder, BioMod achieves state-of-the-art performance across cell-type annotation, zero-shot batch integration, perturbation-response prediction and drug-response forecasting while maintaining a compact architecture that enables efficient inference on a single GPU. Module-level analyses further demonstrate that BioMod captures biologically meaningful cellular programs and preserves pathway-level organization.
Related
- Towards a foundation model for raw nanopore signal through discrete tokenization
- Rethinking One Gene, One Token: Transcriptome-Aware Tokenization for Single-Cell Foundation Models
- Representing biological concepts as directions in foundation model embeddings
- Predictive dendrites as a foundation for biological learning
- Towards Trustworthy Biological Alignment in TabPFN-Probed Pathology Foundation Models
BibTeX
@misc{meng2026biomod,
title = {{BioMod: Biological Module Tokenization for Single-Cell Foundation Modeling}},
author = {L. Meng and J. Zhu and B. Han and Y. Zang and W. Li and K. Zong and J. Zhong and Y. Yang and J. Liu and J. Jia and R. Sun and G. Yu and J. Han and Y. Chen and L. Zhang and J. Xu},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.12.758490},
url = {https://www.biorxiv.org/content/10.64898/2026.10.12.758490v1},
note = {CC BY},
}
RIS
TY - UNPB
TI - BioMod: Biological Module Tokenization for Single-Cell Foundation Modeling
AU - L. Meng
AU - J. Zhu
AU - B. Han
AU - Y. Zang
AU - W. Li
AU - K. Zong
AU - J. Zhong
AU - Y. Yang
AU - J. Liu
AU - J. Jia
AU - R. Sun
AU - G. Yu
AU - J. Han
AU - Y. Chen
AU - L. Zhang
AU - J. Xu
PY - 2026
DA - 2026/10/12
PB - bioRxiv
DO - 10.64898/2026.10.12.758490
UR - https://www.biorxiv.org/content/10.64898/2026.10.12.758490v1
AB - Single-cell foundation models typically represent cells as flat sequences of individual genes, requiring higher-order biological organization to be inferred implicitly from sparse expression profiles. Similar to the evolution from character-level to semantic tokenization in natural language processing,we hypothesize that cellular states can be more efficiently represented using biologically meaningful intermediate units. Here we present BioMod, a 50-million-parameter single-cell foundation model that introduces biological modules as representation units between genes and cells. BioMod constructs an overlapping vocabulary of 660 biological modules derived from curated pathways and data-driven gene programs, replacing thousands of gene-level inputs with compact module representations. Pretrained on approximately 73 million human cells using high-ratio module masking with an asymmetric decoder, BioMod achieves state-of-the-art performance across cell-type annotation, zero-shot batch integration, perturbation-response prediction and drug-response forecasting while maintaining a compact architecture that enables efficient inference on a single GPU. Module-level analyses further demonstrate that BioMod captures biologically meaningful cellular programs and preserves pathway-level organization.
ER -