VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression
Abstract
Spatial transcriptomics can resolve gene expression at single-cell resolution, but it is costly, limited to targeted panels of a few hundred to a few thousand genes, and applicable to only a small number of samples. H&E imaging, by contrast, is cheap and collected routinely at scale. This makes predicting single-cell expression directly from morphology a practical way to bring molecular analysis to large tissue archives. We therefore present VOICE, a multimodal foundation model that predicts single-cell gene expression from H&E images using paired Xenium data. VOICE first aligns cell centered H&E morphology from a pathology foundation model with single-cell expression embeddings from a transcriptome foundation model, trained using contrastive learning over 23 million cells. Next it predicts expression through two branches. One branch directly regresses expression from morphology. The other branch retrieves measured expression from similar reference cells, recovering genes that do not have morphological signal. Because genes vary in morphological predictability, VOICE fuses the two branches with a per-gene weight. After training, VOICE generalizes to heldout patients, slides, and partially overlapping gene panels from Xenium, and it consistently outperforms prior single-cell expression prediction methods on seven metrics.
Related
- Mendel, a foundation model of human genetic variation, prioritizes regulatory variants and improves gene expression prediction
- Mendel, a foundation model of human genetic variation, prioritizes regulatory variants and improves gene expression prediction
- GEO-BERT: A Retrieval augmented LLM Framework for Intelligent Dataset Retrieval from the Gene Expression Omnibus
- RootQuantV2: Adapting a Vision Foundation Model for Root-Trait Regression from Minirhizotron Imagery
- QProtFuse: Query-Guided Layer Fusion for Retrieval-Augmented Protein Function Prediction
BibTeX
@misc{luo2026voice,
title = {{VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression}},
author = {Xin Luo and Yicheng Tao and Haoxuan Zeng and Suyuan Wang and Chenzi Ouyang and Meiqi Zhu and Kai Liu and Shuibing Chen and Jie Liu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2608.08366},
eprint = {2608.08366},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.08366},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - VOICE: A Vision-Omics Foundation Model Integrating Direct and Retrieval-Based Prediction of In-situ Single-Cell Gene Expression
AU - Xin Luo
AU - Yicheng Tao
AU - Haoxuan Zeng
AU - Suyuan Wang
AU - Chenzi Ouyang
AU - Meiqi Zhu
AU - Kai Liu
AU - Shuibing Chen
AU - Jie Liu
PY - 2026
DA - 2026/10/31
PB - arXiv
DO - 10.48550/arxiv.2608.08366
UR - https://arxiv.org/abs/2608.08366
AB - Spatial transcriptomics can resolve gene expression at single-cell resolution, but it is costly, limited to targeted panels of a few hundred to a few thousand genes, and applicable to only a small number of samples. H&E imaging, by contrast, is cheap and collected routinely at scale. This makes predicting single-cell expression directly from morphology a practical way to bring molecular analysis to large tissue archives. We therefore present VOICE, a multimodal foundation model that predicts single-cell gene expression from H&E images using paired Xenium data. VOICE first aligns cell centered H&E morphology from a pathology foundation model with single-cell expression embeddings from a transcriptome foundation model, trained using contrastive learning over 23 million cells. Next it predicts expression through two branches. One branch directly regresses expression from morphology. The other branch retrieves measured expression from similar reference cells, recovering genes that do not have morphological signal. Because genes vary in morphological predictability, VOICE fuses the two branches with a per-gene weight. After training, VOICE generalizes to heldout patients, slides, and partially overlapping gene panels from Xenium, and it consistently outperforms prior single-cell expression prediction methods on seven metrics.
ER -