Unified cell segmentation for spatial transcriptomics via transcript reassignment
- 1University of Illinois Chicago
Abstract
High-resolution spatial transcriptomics (ST) measures gene expression as individually localized molecules on imaging-based platforms or as counts in subcellular bins on sequencing-based platforms, and both must be assigned to cells before cell-level analysis. Existing segmentation methods focus on delineating cell boundaries, yet transcripts that diffuse beyond their cell of origin, fall where cells overlap, or lie in bins that cells only partially cover are still misassigned. Here we present Tracel, a two-stage optimal transport framework that assigns spatial observations to cells under the guidance of a matched single-cell RNA-sequencing reference. Tracel first aligns reference cells to spatial cells to estimate the expected expression profile at each cell location, and then maps transcripts to cells against these profiles. The same formulation assigns transcripts from imaging-based platforms (e.g., Xenium, CosMx) and bin counts from sequencing-based platforms (e.g., Visium HD). Across five human datasets spanning three platforms and three tissues, Tracel improves the separation of cell types over existing methods, assigning cell-type markers to their correct cells, retaining more cell-intrinsic transcripts and reducing contamination from neighboring cells. Tracel also preserves variation within cell types, yielding more accurate differential expression across spatial locations, and its expression profiles attribute ligand-receptor interactions to the correct cell-type pairs in cell-cell communication analysis. Tracel can enter the segmentation workflow at different stages, either refining an existing segmentation or segmenting cells starting from nucleus segmentation.
Related
- MCseg: AI agent-guided workflow search for no-code cell segmentation and transcript attribution in spatial transcriptomics
- Imaging-based spatial isoform transcriptomics resolves transcript diversity at cellular and subcellular resolution
- Agentic AI enables cell segmentation-free cell-type inference and cell-type-guided cell segmentation in spatial transcriptomics
- CellART: a unified framework for extracting single-cell information from high-resolution spatial transcriptomics
- Why rescue-based transcript ranking can mislead: normalization displacement and shared-control coupling in perturbation transcriptomics
BibTeX
@misc{zhang2026unified,
title = {{Unified cell segmentation for spatial transcriptomics via transcript reassignment}},
author = {J. Zhang and X. Cheng and J. Wang and W. Chen and H. Chen},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.03.764126},
url = {https://www.biorxiv.org/content/10.64898/2026.11.03.764126v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Unified cell segmentation for spatial transcriptomics via transcript reassignment
AU - J. Zhang
AU - X. Cheng
AU - J. Wang
AU - W. Chen
AU - H. Chen
PY - 2026
DA - 2026/11/03
PB - bioRxiv
DO - 10.64898/2026.11.03.764126
UR - https://www.biorxiv.org/content/10.64898/2026.11.03.764126v1
AB - High-resolution spatial transcriptomics (ST) measures gene expression as individually localized molecules on imaging-based platforms or as counts in subcellular bins on sequencing-based platforms, and both must be assigned to cells before cell-level analysis. Existing segmentation methods focus on delineating cell boundaries, yet transcripts that diffuse beyond their cell of origin, fall where cells overlap, or lie in bins that cells only partially cover are still misassigned. Here we present Tracel, a two-stage optimal transport framework that assigns spatial observations to cells under the guidance of a matched single-cell RNA-sequencing reference. Tracel first aligns reference cells to spatial cells to estimate the expected expression profile at each cell location, and then maps transcripts to cells against these profiles. The same formulation assigns transcripts from imaging-based platforms (e.g., Xenium, CosMx) and bin counts from sequencing-based platforms (e.g., Visium HD). Across five human datasets spanning three platforms and three tissues, Tracel improves the separation of cell types over existing methods, assigning cell-type markers to their correct cells, retaining more cell-intrinsic transcripts and reducing contamination from neighboring cells. Tracel also preserves variation within cell types, yielding more accurate differential expression across spatial locations, and its expression profiles attribute ligand-receptor interactions to the correct cell-type pairs in cell-cell communication analysis. Tracel can enter the segmentation workflow at different stages, either refining an existing segmentation or segmenting cells starting from nucleus segmentation.
ER -