spOT-NMF: gene expression deconvolution via optimal transport with proximity-aware niche mapping for spatial transcriptomics
Abstract
Spatial transcriptomics enables direct profiling of cellular organization within intact tissues, yet two challenges limit its full exploitation: accurate reference-free deconvolution of mixed expression signals (including species-admixed models such as xenografts), and lack of tools that can translate per-spot program usage values into spatial niches. We developed spOT-NMF, an optimal transport-based non-negative matrix factorization method that replaces standard reconstruction loss with the Wasserstein distance to disentangle mixed signals without requiring matched single-cell atlases. While the generalized OT-NMF factorization is applicable to any expression matrix, a proximity-based niche inference module further translates program co-usage from spatial transcriptomes into directional co-occurrence networks that define tissue compartments and their boundaries. Benchmarking across five gold-standard datasets spanning sub-cellular to multi-cellular resolutions, multiple platforms, and two-species admixture scenarios, spOT-NMF ranks first among reference-free methods, with particular advantage under the signal imbalance conditions characteristic of xenografts. We show that reference-free deconvolution recovers cell populations absent from matched single-cell atlases, highlighting the value of unsupervised analysis. Applied to glioblastoma xenografts and human colorectal cancer, spOT-NMF reveals spatially restricted interferon signaling, invasion routes, and CAF–tumor boundaries. spOT-NMF is available as an open-source Python package with GPU, CPU, and HPC support.
Related
- OT-knn: A neighborhood-aware optimal transport framework for aligning spatial transcriptomics data
- WDGTA: a wavelet-denoised graph attention framework for spatial transcriptomics deconvolution.
- Systematic evaluation of robustness of deconvolution methods for spatial transcriptomics data in case of cell type mismatch.
- Enhancing cell type deconvolution in spatial transcriptomics through incorporating multiple reference panels
- Efficient differential expression analysis of large-scale single-cell transcriptomics data using Dreamlet
BibTeX
@article{abdelkareem2026spotnmf,
title = {{spOT-NMF: gene expression deconvolution via optimal transport with proximity-aware niche mapping for spatial transcriptomics}},
author = {Aly O Abdelkareem and Gurveer S Gill and Varsha Thoppey Manoharan and Theodore B Verhey and A Sorana Morrissy},
journal = {Briefings in Bioinformatics},
year = {2026},
month = nov,
doi = {10.1093/bib/bbag606},
url = {https://doi.org/10.1093/bib/bbag606},
note = {CC BY-NC 4.0},
}
RIS
TY - JOUR
TI - spOT-NMF: gene expression deconvolution via optimal transport with proximity-aware niche mapping for spatial transcriptomics
AU - Aly O Abdelkareem
AU - Gurveer S Gill
AU - Varsha Thoppey Manoharan
AU - Theodore B Verhey
AU - A Sorana Morrissy
PY - 2026
DA - 2026/11/05
JO - Briefings in Bioinformatics
DO - 10.1093/bib/bbag606
UR - https://doi.org/10.1093/bib/bbag606
AB - Spatial transcriptomics enables direct profiling of cellular organization within intact tissues, yet two challenges limit its full exploitation: accurate reference-free deconvolution of mixed expression signals (including species-admixed models such as xenografts), and lack of tools that can translate per-spot program usage values into spatial niches. We developed spOT-NMF, an optimal transport-based non-negative matrix factorization method that replaces standard reconstruction loss with the Wasserstein distance to disentangle mixed signals without requiring matched single-cell atlases. While the generalized OT-NMF factorization is applicable to any expression matrix, a proximity-based niche inference module further translates program co-usage from spatial transcriptomes into directional co-occurrence networks that define tissue compartments and their boundaries. Benchmarking across five gold-standard datasets spanning sub-cellular to multi-cellular resolutions, multiple platforms, and two-species admixture scenarios, spOT-NMF ranks first among reference-free methods, with particular advantage under the signal imbalance conditions characteristic of xenografts. We show that reference-free deconvolution recovers cell populations absent from matched single-cell atlases, highlighting the value of unsupervised analysis. Applied to glioblastoma xenografts and human colorectal cancer, spOT-NMF reveals spatially restricted interferon signaling, invasion routes, and CAF–tumor boundaries. spOT-NMF is available as an open-source Python package with GPU, CPU, and HPC support.
SN - 1467-5463
SN - 1477-4054
ER -