Development and cross-cohort evaluation of a multimodal deep-learning model for ER, PR and HER2 status classification in breast cancer
- 1Hanoi Medical University
Abstract
Background. Estrogen receptor (ER), progesterone receptor (PR) and HER2 status guide systemic treatment in breast cancer. Multimodal models are rarely benchmarked against key biomarker molecular baselines or evaluated for cross-cohort calibration. We assessed incremental value, dependence on label-proximal HER2 transcripts, and transport of discrimination and calibration. Methods. A model combining whole-slide images, transcriptomes, age and stage was developed in TCGA-BRCA (1051 patients; ER/PR/HER2 analysis sets: 1001/998/1006) and evaluated in CPTAC-BRCA (120 patients; 118/113/95). CPTAC had been observed during earlier development and included 13 HER2-positive cases. Analyses included paired model comparisons, transcript removal, alternative loss functions and recalibration. Internal discrimination used pooled out-of-fold predictions; external discrimination used fold-mean estimates, with ensemble calibration reported separately. Internal calibration and macro-AUC uncertainty were interpreted as exploratory. Results. Internal AUCs were 0.943 for ER, 0.901 for PR, and 0.860 for HER2; external AUCs were 0.952, 0.937, and 0.918, respectively. External macro-AUC was 0.936 for the multimodal model and 0.957 for logistic regression using ESR1, PGR, and ERBB2 (difference -0.021). Removal of the extended 29-transcript 17q12-q21 region changed HER2 AUC by -0.035 [-0.058 to -0.013] (95% CI) internally and by -0.042 [-0.089 to -0.007] externally relative to the baseline multimodal model. For external 15-model ensemble predictions, recalibrated HER2 expected calibration error (ECE) was 0.236, exceeding the ECE reference ceiling of 0.125, while calibration-in-the-large was -1.657 [-2.373 to -1.023] (profile-likelihood 95% CI), indicating systematic overprediction. ER and PR also showed overprediction. In paired fold-mean comparisons, HER2 ECE and Brier score favoured the three-transcript model (ESR1, PGR, and ERBB2). Conclusions. A multimodal deep-learning model achieved high discrimination across cohorts but did not demonstrate clear incremental value over a parsimonious three-transcript baseline, while cross-cohort calibration remained a substantial challenge. These findings highlight the importance of evaluating incremental value, calibration, and clinical utility alongside discrimination when assessing AI-based prediction models for medical use.
Related
- Reproducibility and Leakage-Controlled Evaluation of Machine-Learning Models for Chronic Kidney Disease Classification
- Development and evaluation of a desktop VR serious game for color composition learning in basic design education
- Deep Learning and Explainable Artificial Intelligence for Infrared-Thermography-Based CMT-Status Classification in Dairy Cows: A Comparative Evaluation in Egyptian Farms.
- MEMOIR-VLM—a multimodal vision-language model for Alzheimer's disease classification and question answering
- Enhancement of gut-microbiome-based classification of prodromal Parkinson's Disease cases by hyperparameter tuning, integration of multimodal data, and meta-analysis.
BibTeX
@misc{tran2026development,
title = {{Development and cross-cohort evaluation of a multimodal deep-learning model for ER, PR and HER2 status classification in breast cancer}},
author = {B. M. Tran and V. A. Nguyen and Q. A. Dang and N. M. H. Nguyen and Q. A. Nguyen and H. N. Tran and D. H. Tran and T. B. V. Nguyen},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.01.761587},
url = {https://www.biorxiv.org/content/10.64898/2026.11.01.761587v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Development and cross-cohort evaluation of a multimodal deep-learning model for ER, PR and HER2 status classification in breast cancer
AU - B. M. Tran
AU - V. A. Nguyen
AU - Q. A. Dang
AU - N. M. H. Nguyen
AU - Q. A. Nguyen
AU - H. N. Tran
AU - D. H. Tran
AU - T. B. V. Nguyen
PY - 2026
DA - 2026/11/01
PB - bioRxiv
DO - 10.64898/2026.11.01.761587
UR - https://www.biorxiv.org/content/10.64898/2026.11.01.761587v1
AB - Background. Estrogen receptor (ER), progesterone receptor (PR) and HER2 status guide systemic treatment in breast cancer. Multimodal models are rarely benchmarked against key biomarker molecular baselines or evaluated for cross-cohort calibration. We assessed incremental value, dependence on label-proximal HER2 transcripts, and transport of discrimination and calibration. Methods. A model combining whole-slide images, transcriptomes, age and stage was developed in TCGA-BRCA (1051 patients; ER/PR/HER2 analysis sets: 1001/998/1006) and evaluated in CPTAC-BRCA (120 patients; 118/113/95). CPTAC had been observed during earlier development and included 13 HER2-positive cases. Analyses included paired model comparisons, transcript removal, alternative loss functions and recalibration. Internal discrimination used pooled out-of-fold predictions; external discrimination used fold-mean estimates, with ensemble calibration reported separately. Internal calibration and macro-AUC uncertainty were interpreted as exploratory. Results. Internal AUCs were 0.943 for ER, 0.901 for PR, and 0.860 for HER2; external AUCs were 0.952, 0.937, and 0.918, respectively. External macro-AUC was 0.936 for the multimodal model and 0.957 for logistic regression using ESR1, PGR, and ERBB2 (difference -0.021). Removal of the extended 29-transcript 17q12-q21 region changed HER2 AUC by -0.035 [-0.058 to -0.013] (95% CI) internally and by -0.042 [-0.089 to -0.007] externally relative to the baseline multimodal model. For external 15-model ensemble predictions, recalibrated HER2 expected calibration error (ECE) was 0.236, exceeding the ECE reference ceiling of 0.125, while calibration-in-the-large was -1.657 [-2.373 to -1.023] (profile-likelihood 95% CI), indicating systematic overprediction. ER and PR also showed overprediction. In paired fold-mean comparisons, HER2 ECE and Brier score favoured the three-transcript model (ESR1, PGR, and ERBB2). Conclusions. A multimodal deep-learning model achieved high discrimination across cohorts but did not demonstrate clear incremental value over a parsimonious three-transcript baseline, while cross-cohort calibration remained a substantial challenge. These findings highlight the importance of evaluating incremental value, calibration, and clinical utility alongside discrimination when assessing AI-based prediction models for medical use.
ER -