A multimodal foundation model to integrate histology with spatial molecular programs

Abstract

Histology and spatial transcriptomics provide complementary views of cancer morphology and molecular organization, yet their spatial correspondence has not been learned at scale. We developed SQUALL, a multimodal foundation model pretrained on histMol, comprising 1.93 billion spatially aligned histology-transcriptomics spots/bins from 3,446 sections across 33 tissues and 12 platforms. By learning spatial morphology-molecular correspondences, SQUALL enables virtual spatial transcriptomics, tumor microenvironment characterization, and molecularly informed histology representations for clinical prediction. SQUALL generalized virtual spatial gene expression profiling across cohorts, cancers, and platforms. Its representations resolve tumor microenvironments including progression-associated tumor trajectories in breast cancer and recurrence-associated immune-excluded regions in ovarian tumor, together with underlying transcriptional programs. Applied to whole-slide histology, SQUALL enabled outcome and treatment-response prediction while revealing interpretable features associated with risk. Together, these results establish large-scale spatially aligned histology-transcriptomics pretraining as a general framework for connecting tissue morphology, molecular programs, microenvironmental organization, and clinical phenotypes.

Slide-level batch structure limits histology-guided supervision of transcriptomic foundation models
U. McConnell, K. Nonchev, V. H. Koelzer +1
bioRxiv30 Sep 2026PreprintCC BYFull textBioinformatics & computational biologybioRxiv
HESTIA: Scalable Multimodal Integration of Histology and High-Resolution Spatial Transcriptomics for Robust Spatial Domain Identification
Z. Zhong, X. Zhu, J. Guo +2
bioRxiv14 May 2026PreprintCC BY-NCFull textBioinformatics & computational biologybioRxiv
Spatially guided translation from histology images to transcriptomic profiles using foundation model-driven contrastive learning
Zi Huai Huang, Ziyang Xu, Pingzhao Hu
PLOS Computational Biology8 Sep 2026Peer-reviewedCC BY 4.0Full textBioinformatics & computational biology Medical imaging & radiologyPLOS
The SWAN atlas: an open multimodal resource combining neuroimaging and high-resolution histology of the human brain in stereotactic space
E. J. Lopes Alho, H. Friedrich, A. T. D. L. Alho +7
bioRxiv12 Oct 2026PreprintCC BYFull textNeurosciencebioRxiv
VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics
Haenara Shin, Dongjoo Lee, Sungwoo Bae +3
arXiv20 Oct 2026PreprintarXiv licenceFull textBioinformatics & computational biologyarXiv

Cite

BibTeX RIS

BibTeX

@misc{zhang2026multimodal,
  title = {{A multimodal foundation model to integrate histology with spatial molecular programs}},
  author = {Z. Zhang and B. Qin and Y. Zhao and Z. Qi and H. Xu and Y. Wang and W. Zheng and J. Dai and A. Chen and N. Wang and L. Nie and P. Zhang and H. Zhang and Y. Zhao and T. Xu and S. Lin and P. Ren and Z. Zhang and L. Xue and X. Xue and Z. Yang and J. Xu and J. Ma and D. Pan and C. Wang and Z. Liu and Y. Meng and Z. Zeng},
  howpublished = {bioRxiv},
  year = {2026},
  month = jun,
  doi = {10.64898/2026.06.01.729028},
  url = {https://www.biorxiv.org/content/10.64898/2026.06.01.729028v2},
  note = {CC BY-NC},
}

RIS

TY  - UNPB
TI  - A multimodal foundation model to integrate histology with spatial molecular programs
AU  - Z. Zhang
AU  - B. Qin
AU  - Y. Zhao
AU  - Z. Qi
AU  - H. Xu
AU  - Y. Wang
AU  - W. Zheng
AU  - J. Dai
AU  - A. Chen
AU  - N. Wang
AU  - L. Nie
AU  - P. Zhang
AU  - H. Zhang
AU  - Y. Zhao
AU  - T. Xu
AU  - S. Lin
AU  - P. Ren
AU  - Z. Zhang
AU  - L. Xue
AU  - X. Xue
AU  - Z. Yang
AU  - J. Xu
AU  - J. Ma
AU  - D. Pan
AU  - C. Wang
AU  - Z. Liu
AU  - Y. Meng
AU  - Z. Zeng
PY  - 2026
DA  - 2026/06/01
PB  - bioRxiv
DO  - 10.64898/2026.06.01.729028
UR  - https://www.biorxiv.org/content/10.64898/2026.06.01.729028v2
AB  - Histology and spatial transcriptomics provide complementary views of cancer morphology and molecular organization, yet their spatial correspondence has not been learned at scale. We developed SQUALL, a multimodal foundation model pretrained on histMol, comprising 1.93 billion spatially aligned histology-transcriptomics spots/bins from 3,446 sections across 33 tissues and 12 platforms. By learning spatial morphology-molecular correspondences, SQUALL enables virtual spatial transcriptomics, tumor microenvironment characterization, and molecularly informed histology representations for clinical prediction. SQUALL generalized virtual spatial gene expression profiling across cohorts, cancers, and platforms. Its representations resolve tumor microenvironments including progression-associated tumor trajectories in breast cancer and recurrence-associated immune-excluded regions in ovarian tumor, together with underlying transcriptional programs. Applied to whole-slide histology, SQUALL enabled outcome and treatment-response prediction while revealing interpretable features associated with risk. Together, these results establish large-scale spatially aligned histology-transcriptomics pretraining as a general framework for connecting tissue morphology, molecular programs, microenvironmental organization, and clinical phenotypes.
ER  -

Abstract

Related

Cite