Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma
- 1Department of Diagnostic Radiology, Hunan Cancer Hospital and the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha, Hunan, People's Republic of China
- 2Hunan Provincial Key Laboratory on Bioinformatics, School of Computer Science and Engineering, Central South University, Changsha, People's Republic of China
- 3Department of Health Service Center, Hunan Cancer Hospital and the Affiliated Cancer Hospital of Xiangya School of Medicine, Central South University, Changsha, Hunan, People's Republic of China
Abstract
Predicting response to induction chemotherapy (IC) and overall survival (OS) is critical for optimizing treatment in patients with locally advanced nasopharyngeal carcinoma (LANPC). This study aimed to develop and validate a multi-task deep learning model integrating pretreatment MRI and whole slide images (WSIs) to predict IC response and OS in LANPC. Pretreatment MRI and WSIs from 404 patients with LANPC were retrospectively collected to construct a multi-task model (MoEMIL) for the simultaneous prediction of early IC response and OS. In our cohort, no progressive disease cases were observed; therefore, the poor responder group consisted exclusively of patients with stable disease. MoEMIL employed multi-instance learning to process WSIs, PyRadiomics and a convolutional neural network (ResNet50) to extract MRI features, and fused multimodal features through a multi-gate mixture-of-experts architecture. Clustering-constrained attention multiple instance learning and gradient-weighted class activation mapping were applied for visualization and interpretation. MoEMIL effectively stratified patients into good and poor IC response groups, achieving areas under the curve of 0.917, 0.869, and 0.801 in the train, validation, and test sets, respectively. In the test set, MoEMIL significantly outperformed the deep learning radiomics model and TNM staging for both endpoints. Compared to the Pathomics model, MoEMIL achieved significant improvement for OS (P = 0.0049) and borderline improvement for IC response (P = 0.064). The model could stratify patients into high- and low-risk OS groups (P < 0.05). MoEMIL might be an adjunctive tool for early induction chemotherapy response prediction and prognostic stratification in LANPC. Author summary: We have developed a deep learning model that integrates two types of medical images, including magnetic resonance imaging (MRI) and digital pathological slices, to simultaneously predict response to induction chemotherapy and prognosis in patients with locally advanced nasopharyngeal carcinoma. Current treatment decisions primarily rely on traditional tumor staging (TNM), which often fails to comprehensively reflect the complexity of the disease. Our model, named MoEMIL, was trained and tested on data from 404 patients across two hospitals and consistently outperformed both single-model approaches and TNM staging methods. By identifying patients who exhibit poor response to induction chemotherapy or higher prognostic risk, our tool may assist clinicians in risk stratification and treatment planning, enabling intensified management for high-risk patients and avoiding unnecessary side effects for low-risk patients, but prospective validation is required before routine clinical adoption. Additionally, we visualize the model’s reasoning process through heat map generation, which highlights the image regions exerting the greatest influence on prediction outcomes. This work represents a step toward more precise treatment for nasopharyngeal carcinoma; however, larger-scale prospective studies are required before the model can be integrated into routine clinical practice.
Related
- Comparative efficacy and toxicity of two versus three induction chemotherapy cycles in locally advanced nasopharyngeal carcinoma: a propensity score matching analysis.
- Efficacy and safety of neoadjuvant chemotherapy with toripalimab in locally advanced nasopharyngeal carcinoma: a phase II trial and biological correlate analysis
- A pretreatment TNM-metabolic vulnerability score with EBV DNA adjustment for risk stratification in non-metastatic nasopharyngeal carcinoma
- Recursive partitioning analysis model based on ¹⁸F-FDG PET/CT metabolic parameters for recurrent or metastatic nasopharyngeal carcinoma treated with PD-1 inhibitors and chemotherapy.
- Induction Chemoimmunotherapy Versus Induction Chemotherapy Followed by Concurrent Chemoradiotherapy for Locally Advanced Esophageal Squamous Cell Carcinoma: A Propensity Score-Matched Study
BibTeX
@article{hou2026multitask,
title = {{Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma}},
author = {Jing Hou and Xiaochun Yi and Congrui Li and Junjian Li and Hui Cao and Qiang Lu and Xiaoping Yu},
journal = {PLOS Digital Health},
year = {2026},
month = oct,
doi = {10.1371/journal.pdig.0001378},
url = {https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001378},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Multi-task deep learning integrating pretreatment MRI and whole slide images predicts induction chemotherapy response and survival in locally advanced nasopharyngeal carcinoma
AU - Jing Hou
AU - Xiaochun Yi
AU - Congrui Li
AU - Junjian Li
AU - Hui Cao
AU - Qiang Lu
AU - Xiaoping Yu
PY - 2026
DA - 2026/10/05
JO - PLOS Digital Health
DO - 10.1371/journal.pdig.0001378
UR - https://journals.plos.org/digitalhealth/article?id=10.1371/journal.pdig.0001378
AB - Predicting response to induction chemotherapy (IC) and overall survival (OS) is critical for optimizing treatment in patients with locally advanced nasopharyngeal carcinoma (LANPC). This study aimed to develop and validate a multi-task deep learning model integrating pretreatment MRI and whole slide images (WSIs) to predict IC response and OS in LANPC. Pretreatment MRI and WSIs from 404 patients with LANPC were retrospectively collected to construct a multi-task model (MoEMIL) for the simultaneous prediction of early IC response and OS. In our cohort, no progressive disease cases were observed; therefore, the poor responder group consisted exclusively of patients with stable disease. MoEMIL employed multi-instance learning to process WSIs, PyRadiomics and a convolutional neural network (ResNet50) to extract MRI features, and fused multimodal features through a multi-gate mixture-of-experts architecture. Clustering-constrained attention multiple instance learning and gradient-weighted class activation mapping were applied for visualization and interpretation. MoEMIL effectively stratified patients into good and poor IC response groups, achieving areas under the curve of 0.917, 0.869, and 0.801 in the train, validation, and test sets, respectively. In the test set, MoEMIL significantly outperformed the deep learning radiomics model and TNM staging for both endpoints. Compared to the Pathomics model, MoEMIL achieved significant improvement for OS (P = 0.0049) and borderline improvement for IC response (P = 0.064). The model could stratify patients into high- and low-risk OS groups (P < 0.05). MoEMIL might be an adjunctive tool for early induction chemotherapy response prediction and prognostic stratification in LANPC. Author summary: We have developed a deep learning model that integrates two types of medical images, including magnetic resonance imaging (MRI) and digital pathological slices, to simultaneously predict response to induction chemotherapy and prognosis in patients with locally advanced nasopharyngeal carcinoma. Current treatment decisions primarily rely on traditional tumor staging (TNM), which often fails to comprehensively reflect the complexity of the disease. Our model, named MoEMIL, was trained and tested on data from 404 patients across two hospitals and consistently outperformed both single-model approaches and TNM staging methods. By identifying patients who exhibit poor response to induction chemotherapy or higher prognostic risk, our tool may assist clinicians in risk stratification and treatment planning, enabling intensified management for high-risk patients and avoiding unnecessary side effects for low-risk patients, but prospective validation is required before routine clinical adoption. Additionally, we visualize the model’s reasoning process through heat map generation, which highlights the image regions exerting the greatest influence on prediction outcomes. This work represents a step toward more precise treatment for nasopharyngeal carcinoma; however, larger-scale prospective studies are required before the model can be integrated into routine clinical practice.
SN - 2767-3170
ER -