Towards a knowledge-enhanced single-cell foundation model
Abstract
Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation and gene-level regulatory information, provided additional scaling dimension than simply increasing data size. Motivated by this observation, we present scKITE, a simple yet effective scFM that integrates cell-annotation and gene-regulatory supervision into a shared transcriptomic Transformer encoder through lightweight auxiliary decoders. These decoders are used only during pretraining and subsequently discarded, yielding a general-purpose encoder enriched with biological knowledge for downstream applications. With only 179,067 pretraining samples, i.e., less than 0.5\% of those used by previous strong scFMs, scKITE outperformed these models across diverse downstream tasks, highlighting knowledge-enhanced pretraining as a promising paradigm for biologically grounded scFMs.
Related
- Where single-cell foundation models help: a benchmark of annotation, perturbation and regulatory inference against simple baselines
- scRM: A single-cell foundation model trained on scRNA-seq reads
- Benchmarking single-cell foundation models reveals an advantage in the low-data regime for cell-type and disease-state prediction
- CellPatch: a flexible and efficient framework for single-cell foundation models empowered by heuristic gene patching
- DECONVersation: Single Cell Foundation Model-Derived Embeddings for Robust Cell Type Deconvolution of Bulk RNA-seq
BibTeX
@misc{zhang2026towards,
title = {{Towards a knowledge-enhanced single-cell foundation model}},
author = {Hanqing Zhang and Jie Bao and Mei Ma and Shuai Liu and Jiaying Ma and Jiaguan Liu and Jiaxiao Li and Zhenbo Li and Wenwen Gong and Zhijun Ca},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.14970},
eprint = {2609.14970},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.14970},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Towards a knowledge-enhanced single-cell foundation model
AU - Hanqing Zhang
AU - Jie Bao
AU - Mei Ma
AU - Shuai Liu
AU - Jiaying Ma
AU - Jiaguan Liu
AU - Jiaxiao Li
AU - Zhenbo Li
AU - Wenwen Gong
AU - Zhijun Ca
PY - 2026
DA - 2026/09/14
PB - arXiv
DO - 10.48550/arxiv.2609.14970
UR - https://arxiv.org/abs/2609.14970
AB - Single-cell foundation models (scFMs) increasingly rely on large-scale transcriptomic pretraining, yet expanding pretraining data can yield diminishing gains while substantially increasing computational cost. Our data scaling analyses showed that incorporating biological knowledge, including cell-level text annotation and gene-level regulatory information, provided additional scaling dimension than simply increasing data size. Motivated by this observation, we present scKITE, a simple yet effective scFM that integrates cell-annotation and gene-regulatory supervision into a shared transcriptomic Transformer encoder through lightweight auxiliary decoders. These decoders are used only during pretraining and subsequently discarded, yielding a general-purpose encoder enriched with biological knowledge for downstream applications. With only 179,067 pretraining samples, i.e., less than 0.5\% of those used by previous strong scFMs, scKITE outperformed these models across diverse downstream tasks, highlighting knowledge-enhanced pretraining as a promising paradigm for biologically grounded scFMs.
ER -