Virtual IHC Synthesis and Bimodal Fusion for Patch-Level Classification of HER2 IHC Expression in Breast Cancer
4 affiliations
- 1The Affiliated Xuzhou Clinical College of Xuzhou Medical University, Xuzhou, Jiangsu, People’s Republic of China
- 2School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, Jiangsu, People’s Republic of China
- 3Department of Radiation Oncology, Southeast University Affiliated Xuzhou Central Hospital, Xuzhou, Jiangsu, People’s Republic of China
- 4Department of Pathology, Southeast University Affiliated Xuzhou Central Hospital, Xuzhou, Jiangsu, People’s Republic of China
Abstract
Purpose: Accurate assessment of the human epidermal growth factor receptor 2 (HER2) biomarker plays a pivotal role in diagnosing breast malignancies and guiding therapeutic selection, yet routine immunohistochemistry (IHC) testing involves high expenses, requires extensive manual effort, and is subject to diagnostic inconsistencies among pathologists. To overcome these limitations, we propose a cascaded two-stage framework for virtual HER2 IHC synthesis and patch-level classification of HER2 IHC expression from routine hematoxylin and eosin (H&E)-stained images. The prediction target comprises the four conventional HER2 IHC expression categories: 0, 1+, 2+, and 3+.
Patients and Methods: The proposed pipeline operates in two distinct phases. First, a progressive domain knowledge-constrained generative model is developed to synthesize virtual HER2 IHC images from H&E inputs by incorporating structural and staining-aware priors. Second, the generated virtual HER2 images are paired with the corresponding H&E patches and jointly processed by a bimodal fusion network with guided purification and feature recalibration for patch-level classification of HER2 IHC expression. Experiments were conducted on a retrospective breast cancer pathology dataset comprising 48 patients with paired H&E and HER2 IHC whole-slide images (WSIs) digitized at 20x magnification, sourced from patients identified at Xuzhou Central Hospital between January 2018 and December 2023, yielding a total of 9,287 paired patches for model training and testing.
Results: The introduced generative framework attains the lowest Fréchet inception distance (FID) of 22.67 among competing methods, indicating superior pathological realism in the synthesized images. In addition, in the four-class patch-level HER2 classification task, the proposed bimodal framework outperforms unimodal H&E-based baselines, yielding an accuracy of 67.12%, an F1-score of 63.63%, and an area under the curve (AUC) of 84.99%.
Conclusion: These results demonstrate that the virtual HER2 modality provides complementary HER2-oriented cues beyond native morphology alone and that the proposed framework shows potential as a computational tool for low-cost HER2 assessment from routine H&E images.
Related
- Boundary-aware multi-scale residual learning with hybrid radiomic–deep fusion for lung tumor segmentation and classification
- MedFuse: dual-stream fusion of convolutional and vision transformer-based features for enhanced medical image classification
- Multimodal fusion of SpO₂ time series and clinical data for explainable classification of obstructive sleep apnea severity
- CSwinFK: a bidirectional cross-attention CNN-Swin Transformer fusion model for fungal keratitis classification in in vivo confocal microscopy images
- Early vs late fusion of qMIDS with machine learning based classifiers for oral cancer diagnosis from gene expression data
BibTeX
@article{tang2026virtual,
title = {{Virtual IHC Synthesis and Bimodal Fusion for Patch-Level Classification of HER2 IHC Expression in Breast Cancer}},
author = {Yi Tang and Keyu Chen and Xinru Yang and Hui Hui and Chengjun Wu and Na Li and Di Wu and Yun Zhou},
journal = {Cancer Management and Research},
year = {2026},
month = oct,
doi = {10.2147/cmar.s647760},
url = {https://doi.org/10.2147/cmar.s647760},
note = {CC BY-NC 4.0},
}
RIS
TY - JOUR
TI - Virtual IHC Synthesis and Bimodal Fusion for Patch-Level Classification of HER2 IHC Expression in Breast Cancer
AU - Yi Tang
AU - Keyu Chen
AU - Xinru Yang
AU - Hui Hui
AU - Chengjun Wu
AU - Na Li
AU - Di Wu
AU - Yun Zhou
PY - 2026
DA - 2026/10/25
JO - Cancer Management and Research
DO - 10.2147/cmar.s647760
UR - https://doi.org/10.2147/cmar.s647760
AB - Purpose: Accurate assessment of the human epidermal growth factor receptor 2 (HER2) biomarker plays a pivotal role in diagnosing breast malignancies and guiding therapeutic selection, yet routine immunohistochemistry (IHC) testing involves high expenses, requires extensive manual effort, and is subject to diagnostic inconsistencies among pathologists. To overcome these limitations, we propose a cascaded two-stage framework for virtual HER2 IHC synthesis and patch-level classification of HER2 IHC expression from routine hematoxylin and eosin (H&E)-stained images. The prediction target comprises the four conventional HER2 IHC expression categories: 0, 1+, 2+, and 3+. Patients and Methods: The proposed pipeline operates in two distinct phases. First, a progressive domain knowledge-constrained generative model is developed to synthesize virtual HER2 IHC images from H&E inputs by incorporating structural and staining-aware priors. Second, the generated virtual HER2 images are paired with the corresponding H&E patches and jointly processed by a bimodal fusion network with guided purification and feature recalibration for patch-level classification of HER2 IHC expression. Experiments were conducted on a retrospective breast cancer pathology dataset comprising 48 patients with paired H&E and HER2 IHC whole-slide images (WSIs) digitized at 20x magnification, sourced from patients identified at Xuzhou Central Hospital between January 2018 and December 2023, yielding a total of 9,287 paired patches for model training and testing. Results: The introduced generative framework attains the lowest Fréchet inception distance (FID) of 22.67 among competing methods, indicating superior pathological realism in the synthesized images. In addition, in the four-class patch-level HER2 classification task, the proposed bimodal framework outperforms unimodal H&E-based baselines, yielding an accuracy of 67.12%, an F1-score of 63.63%, and an area under the curve (AUC) of 84.99%. Conclusion: These results demonstrate that the virtual HER2 modality provides complementary HER2-oriented cues beyond native morphology alone and that the proposed framework shows potential as a computational tool for low-cost HER2 assessment from routine H&E images.
SN - 1179-1322
ER -