Transformer-Based Multimodal Fusion Integrating Computed Tomography, Pathology, and Clinical Features for Oral Squamous Cell Carcinoma Recurrence Prediction
- 1Department Oral & Maxillofacial Surgery, The Second Hospital of Dalian Medical University, Dalian Medical University, Dalian, Liaoning, China
- 2Department Oral & Maxillofacial Surgery, The First Hospital of Dalian Medical University, Dalian Medical University, Dalian, Liaoning, China
Abstract
Background: Postoperative recurrence is the leading cause of treatment failure in oral squamous cell carcinoma (OSCC), yet current risk-stratification schemes offer only modest predictive performance. We developed a Transformer-based multimodal fusion framework integrating computed tomography (CT), histopathology, and clinical variables for individualized recurrence prediction. Methods: A total of 134 patients with OSCC were retrospectively enrolled. Using a dual-stream strategy, deep semantic representations were derived from preoperative contrast-enhanced CT and haematoxylin and eosin-stained histopathological images via MnasNet-0.5, while quantitative radiomic descriptors were extracted using PyRadiomics. Together with structured clinical variables, these constituted five parallel feature streams, which – after dimensional harmonization – were passed through a three-layer Transformer encoder employing self-attention and learnable modality weights for nonlinear cross-modal integration. Performance was assessed by fivefold stratified cross-validation. Results: The full-modality TransformerFusion model achieved an AUC of 0.909 ± 0.069, with a sensitivity of 0.791, specificity of 0.834, and precision of 0.854. Decision curve analysis demonstrated superior net clinical benefit over logistic regression and the treat-none strategy across threshold probabilities of 0.20 to 0.70. Ablation experiments revealed that the pathology dual-stream, CT dual-stream, and clinical-only submodels attained AUCs of 0.840, 0.751, and 0.773, respectively – each inferior to the integrated model. Self-attention-based fusion (AUC = 0.909) also outperformed feature concatenation (AUC = 0.854). Conclusion: The proposed framework captures complementary prognostic signals across CT, histopathology, and clinical variables, outperforming single-modality counterparts in recurrence prediction. As it relies solely on routine preoperative CT and standard haematoxylin and eosin slides, it offers a clinically deployable tool for postoperative risk stratification in OSCC.
Related
- Transformer-Based Deep Learning Integrating Multi-Regional MRI and Clinical Features for Predicting Biochemical Recurrence After Radical Prostatectomy: A Multicenter Study.
- Nomogram models integrating multiregion radiomics, quantitative computed tomography, and clinical indicators for distinguishing connective tissue disease-associated interstitial lung disease from idiopathic pulmonary fibrosis and improving the prediction of gender-age-physiology stage
- Application of a multimodal model integrating optical coherence tomography angiography (OCTA) and radiomics for the identification of high-risk Nonarteritic anterior ischaemic optic neuropathy (NAION) populations and for distinguishing its acute phase.
- Three-dimensional multimodal fractal dynamics integrating spatial metabolism, cytoarchitecture, and perfusion for preoperative prediction of microvascular invasion in colorectal liver metastases
- Multi-Scale MRI, Radiology Reports, and Blood Biomarker–Guided Multimodal Deep Learning for Predicting Postoperative Recurrence in Cervical Cancer: A Multicenter Study
BibTeX
@article{yang2026transformerbased,
title = {{Transformer-Based Multimodal Fusion Integrating Computed Tomography, Pathology, and Clinical Features for Oral Squamous Cell Carcinoma Recurrence Prediction}},
author = {Shuwen Yang and Fuyong Sui and Zhuang Liang and Qilin Liu and Binyang Fu and Hui Dong and Zhen Wang},
journal = {International Dental Journal},
year = {2026},
month = oct,
doi = {10.1016/j.identj.2026.111098},
url = {https://doi.org/10.1016/j.identj.2026.111098},
note = {CC BY},
}
RIS
TY - JOUR
TI - Transformer-Based Multimodal Fusion Integrating Computed Tomography, Pathology, and Clinical Features for Oral Squamous Cell Carcinoma Recurrence Prediction
AU - Shuwen Yang
AU - Fuyong Sui
AU - Zhuang Liang
AU - Qilin Liu
AU - Binyang Fu
AU - Hui Dong
AU - Zhen Wang
PY - 2026
DA - 2026/10/31
JO - International Dental Journal
DO - 10.1016/j.identj.2026.111098
UR - https://doi.org/10.1016/j.identj.2026.111098
AB - Background: Postoperative recurrence is the leading cause of treatment failure in oral squamous cell carcinoma (OSCC), yet current risk-stratification schemes offer only modest predictive performance. We developed a Transformer-based multimodal fusion framework integrating computed tomography (CT), histopathology, and clinical variables for individualized recurrence prediction. Methods: A total of 134 patients with OSCC were retrospectively enrolled. Using a dual-stream strategy, deep semantic representations were derived from preoperative contrast-enhanced CT and haematoxylin and eosin-stained histopathological images via MnasNet-0.5, while quantitative radiomic descriptors were extracted using PyRadiomics. Together with structured clinical variables, these constituted five parallel feature streams, which – after dimensional harmonization – were passed through a three-layer Transformer encoder employing self-attention and learnable modality weights for nonlinear cross-modal integration. Performance was assessed by fivefold stratified cross-validation. Results: The full-modality TransformerFusion model achieved an AUC of 0.909 ± 0.069, with a sensitivity of 0.791, specificity of 0.834, and precision of 0.854. Decision curve analysis demonstrated superior net clinical benefit over logistic regression and the treat-none strategy across threshold probabilities of 0.20 to 0.70. Ablation experiments revealed that the pathology dual-stream, CT dual-stream, and clinical-only submodels attained AUCs of 0.840, 0.751, and 0.773, respectively – each inferior to the integrated model. Self-attention-based fusion (AUC = 0.909) also outperformed feature concatenation (AUC = 0.854). Conclusion: The proposed framework captures complementary prognostic signals across CT, histopathology, and clinical variables, outperforming single-modality counterparts in recurrence prediction. As it relies solely on routine preoperative CT and standard haematoxylin and eosin slides, it offers a clinically deployable tool for postoperative risk stratification in OSCC.
SN - 0020-6539
SN - 1875-595X
ER -