Fine-Grained and Reliable Cell Classification from Pathology Images
- 1EPFL
Abstract
Molecular assays such as spatial transcriptomics facilitate fine-grained identification of cell types for analyzing tumor microenvironments. However, they are expensive to acquire in routine clinical practice compared to hematoxylin and eosin (H&E) images. With molecular guidance, such images can distinguish cell lineages, yet often fail to resolve closely related subtypes. To overcome this, we introduce CellBridge, a multi-modal contrastive learning framework pretrained on 16M cells with paired H&E and spatial profiles, for transferring molecular information into morphological representations. CellBridge further incorporates a Cell-Ontology-aware conformal prediction module that assigns labels at different levels of specificity: leaf-level cell types when morphology alone provides sufficient evidence, and broader ancestor categories when distinguishing among more specific cell types would require molecular information. Across in-domain and out-of-domain benchmarks, CellBridge outperforms leading pathology foundation models in macro F1 by up to 16 percentage points, with the largest gains on the most fine-grained label spaces. It consistently achieves smaller prediction sets, while reducing class-wise coverage gaps. We envision CellBridge as a step toward extracting cellular information that currently requires costly molecular assays from routinely acquired pathology images, while explicitly identifying where such assays remain necessary. Our code is available at https://github.com/Lotfollahi-lab/cellbridge.
Related
- Fine-grained hierarchical crop type classification from integrated hyperspectral EnMAP data and multispectral sentinel-2 time series: A large-scale dataset and dual-stream transformer method
- Fine-grained sugarcane varietal classification using high-resolution PRSS-1 satellite imagery: a field-validated dataset and a spatial context-aware deep learning framework
- Human similarity judgments reveal fine-grained representations of dynamic social scenes
- FMCerNet: frequency-aware multi-label learning for fine-grained cervical cytology screening
- TPE-optimized XGBoost model for fine–grained prediction of tertiary vocational college students’ employment destinations
BibTeX
@misc{ogut2026finegrained,
title = {{Fine-Grained and Reliable Cell Classification from Pathology Images}},
author = {S. Ogut and A. Merchant and A. Vahidi and D. Jeong and S. Birk and A. Boxall and A. Miraki Feriz and M. R. Sallese and Y. Chang and P. Frossard and D. Thanou and M. Lotfollahi},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.20.760347},
url = {https://www.biorxiv.org/content/10.64898/2026.10.20.760347v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Fine-Grained and Reliable Cell Classification from Pathology Images
AU - S. Ogut
AU - A. Merchant
AU - A. Vahidi
AU - D. Jeong
AU - S. Birk
AU - A. Boxall
AU - A. Miraki Feriz
AU - M. R. Sallese
AU - Y. Chang
AU - P. Frossard
AU - D. Thanou
AU - M. Lotfollahi
PY - 2026
DA - 2026/10/20
PB - bioRxiv
DO - 10.64898/2026.10.20.760347
UR - https://www.biorxiv.org/content/10.64898/2026.10.20.760347v1
AB - Molecular assays such as spatial transcriptomics facilitate fine-grained identification of cell types for analyzing tumor microenvironments. However, they are expensive to acquire in routine clinical practice compared to hematoxylin and eosin (H&E) images. With molecular guidance, such images can distinguish cell lineages, yet often fail to resolve closely related subtypes. To overcome this, we introduce CellBridge, a multi-modal contrastive learning framework pretrained on 16M cells with paired H&E and spatial profiles, for transferring molecular information into morphological representations. CellBridge further incorporates a Cell-Ontology-aware conformal prediction module that assigns labels at different levels of specificity: leaf-level cell types when morphology alone provides sufficient evidence, and broader ancestor categories when distinguishing among more specific cell types would require molecular information. Across in-domain and out-of-domain benchmarks, CellBridge outperforms leading pathology foundation models in macro F1 by up to 16 percentage points, with the largest gains on the most fine-grained label spaces. It consistently achieves smaller prediction sets, while reducing class-wise coverage gaps. We envision CellBridge as a step toward extracting cellular information that currently requires costly molecular assays from routinely acquired pathology images, while explicitly identifying where such assays remain necessary. Our code is available at https://github.com/Lotfollahi-lab/cellbridge.
ER -