RNA-guided contrastive learning enhances patient-level prediction from histology
- 1Boston University Chobanian & Avedisian School of Medicine
Abstract
Predicting molecular receptor status, including estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2), directly from routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) could reduce reliance on costly immunohistochemistry, but image-only models have no access to the transcriptomic programs that define these subtypes. Here we show that coupling a whole-slide vision transformer (GigaPath) with bulk RNA-sequencing profiles through a training-time-only contrastive objective improves receptor-status classification from WSIs alone, without requiring any RNA-seq data at inference. Rather than training a dedicated bulk-transcriptomics encoder, we purpose the gene-sentence encoding and pretrained text encoder from BioBERT1 and apply them to bulk RNA-seq profiles from The Cancer Genome Atlas breast cancer cohort (TCGA-BRCA). Under five-fold cross-validation, this RNA-guided pre-training consistently improved AUROC for ER, PR, and HER2 relative to a contrastive-pre-training-free baseline and reproducibly introduced receptor-relevant structure into the learned slide representation. However, evaluation on independent external cohorts revealed a striking, receptor-dependent divergence: gains for ER and PR, both governed by broad, multi-gene luminal transcriptional programs, generalized robustly, whereas the apparent AUROC improvement for HER2, driven by focal ERBB2 amplification rather than a coordinated transcriptional signature, masked a severe loss of classification sensitivity at the operating threshold. These findings establish training-time transcriptomic supervision as a practical route to enhancing histology-only molecular subtyping, while showing that gene-selection strategy, and not model architecture alone, determines which molecular alterations such supervision can teach a vision model to recognize.
Related
- Spatially guided translation from histology images to transcriptomic profiles using foundation model-driven contrastive learning
- Contrastive learning and advanced bundle metrics outperform fractional anisotropy for Alzheimer’s disease diffusion MRI prediction: findings from the Beyond FA challenge
- PRInter: an inductive multimodal heterogeneous network embedding framework with contrastive learning for lncRNA-protein interaction prediction
- Prediction of Spinal Pain Intensity from Perception-Dependent Stimulation Thresholds and Patient Characteristics: A Regression and Machine Learning Approach
- Modeling Factors Associated with Smoking Dependence Severity Using Binary Classification-Based Machine Learning: Prediction with Hematological and Demographic Patient Data
BibTeX
@misc{zou2026rnaguided,
title = {{RNA-guided contrastive learning enhances patient-level prediction from histology}},
author = {A. Zou and E. Huang and Q. Wu and M. E. Barnard and C. Zhang},
howpublished = {bioRxiv},
year = {2026},
month = sep,
doi = {10.64898/2026.09.24.754149},
url = {https://www.biorxiv.org/content/10.64898/2026.09.24.754149v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - RNA-guided contrastive learning enhances patient-level prediction from histology
AU - A. Zou
AU - E. Huang
AU - Q. Wu
AU - M. E. Barnard
AU - C. Zhang
PY - 2026
DA - 2026/09/24
PB - bioRxiv
DO - 10.64898/2026.09.24.754149
UR - https://www.biorxiv.org/content/10.64898/2026.09.24.754149v1
AB - Predicting molecular receptor status, including estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2), directly from routine hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) could reduce reliance on costly immunohistochemistry, but image-only models have no access to the transcriptomic programs that define these subtypes. Here we show that coupling a whole-slide vision transformer (GigaPath) with bulk RNA-sequencing profiles through a training-time-only contrastive objective improves receptor-status classification from WSIs alone, without requiring any RNA-seq data at inference. Rather than training a dedicated bulk-transcriptomics encoder, we purpose the gene-sentence encoding and pretrained text encoder from BioBERT1 and apply them to bulk RNA-seq profiles from The Cancer Genome Atlas breast cancer cohort (TCGA-BRCA). Under five-fold cross-validation, this RNA-guided pre-training consistently improved AUROC for ER, PR, and HER2 relative to a contrastive-pre-training-free baseline and reproducibly introduced receptor-relevant structure into the learned slide representation. However, evaluation on independent external cohorts revealed a striking, receptor-dependent divergence: gains for ER and PR, both governed by broad, multi-gene luminal transcriptional programs, generalized robustly, whereas the apparent AUROC improvement for HER2, driven by focal ERBB2 amplification rather than a coordinated transcriptional signature, masked a severe loss of classification sensitivity at the operating threshold. These findings establish training-time transcriptomic supervision as a practical route to enhancing histology-only molecular subtyping, while showing that gene-selection strategy, and not model architecture alone, determines which molecular alterations such supervision can teach a vision model to recognize.
ER -