MIFF-CLIP-HCC: Contrastive Language-Image Pretraining-enhanced multimodal information fusion framework for early recurrence prediction in postoperative hepatocellular carcinoma.
- 1Medical AI Lab, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen, China.
- 2Department of Radiology, The First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China.
- 3Department of Radiology, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Collaborative Innovation Center for Cancer Medicine, Guangzhou, China.
Abstract
Hepatocellular carcinoma (HCC) is one of the most common malignant tumors in China, and its prognosis is affected by many factors, such as tumor stage, liver function status, and patient health. To formulate treatment plans and evaluate prognosis, clinicians require accurate prediction of early postoperative recurrence of HCC. To address this clinical need, this study proposes a novel deep learning framework, termed MIFF-CLIP-HCC, to predict the early postoperative recurrence of HCC based on multimodal information fusion framework (MIFF) and Contrastive Language-Image Pretraining (CLIP). The MIFF-CLIP-HCC framework comprises two core steps. It first uses a fusion framework with a bidirectional attention mechanism to integrate multimodal information, including arterial phase images, venous phase images, and clinical data from patients with HCC. It then introduces image features extracted by the foundational CLIP model to deeply mine effective information from arterial and venous phase images, which increases the performance of the model. Between 2022 and 2025, a total of 240 early-stage HCC patients from two centers were included. Of these, 180 were from The First Affiliated Hospital of Sun Yat-sen University (Center 1) (including 114 early recurrence cases and 66 non-early recurrence cases, with an 8:2 ratio for training and internal testing), and the remaining 60 were from Sun Yat-sen University Cancer Center (Center 2) (including 35 early recurrence cases and 25 non-early recurrence cases), designated as the external test cohort. The accuracy and robustness of the model were validated on internal and external test sets. The model achieved area under the receiver operating characteristic curve (AUC) values of 0.844 [95% confidence interval (CI): 0.726-0.963] on the internal test set and 0.759 (95% CI: 0.651-0.867) on the external test set, outperforming other fusion frameworks (best baseline AUC: 0.779 on internal and 0.690 on external). The results demonstrate that the proposed MIFF-CLIP-HCC framework, which employs dual optimization strategies of multimodal information fusion and CLIP, significantly improves the accuracy of early postoperative recurrence prediction for HCC, thus demonstrating considerable potential for clinical application.
Related
- PRInter: an inductive multimodal heterogeneous network embedding framework with contrastive learning for lncRNA-protein interaction prediction
- Preoperative Zinc Deficiency as a Novel Prognostic Predictor for Postoperative Recurrence in Hepatocellular Carcinoma
- Multi-Scale MRI, Radiology Reports, and Blood Biomarker–Guided Multimodal Deep Learning for Predicting Postoperative Recurrence in Cervical Cancer: A Multicenter Study
- Performance and incremental value of preoperative multimodal imaging and perioperative clinical characteristics in predicting postoperative major adverse events in type A aortic dissection
- Preoperative FIB‐4 Index as a Potential Factor Associated With Severe Postoperative Complications and Endogenous Organ Failure After Hepatectomy for Hepatocellular Carcinoma
BibTeX
@article{diao2026miffcliphcc,
title = {{MIFF-CLIP-HCC: Contrastive Language-Image Pretraining-enhanced multimodal information fusion framework for early recurrence prediction in postoperative hepatocellular carcinoma.}},
author = {Xianfen Diao and Ruihao Liu and Boxian Li and Zhaoyang Lu and Zhenhua Gao and Shiting Feng and Jifei Wang and Chuanmiao Xie and Meili Sun and Bingsheng Huang},
journal = {Quantitative imaging in medicine and surgery},
year = {2026},
month = oct,
doi = {10.21037/qims-2026-0513},
url = {https://europepmc.org/article/MED/42997797},
note = {CC BY-NC-ND},
}
RIS
TY - JOUR
TI - MIFF-CLIP-HCC: Contrastive Language-Image Pretraining-enhanced multimodal information fusion framework for early recurrence prediction in postoperative hepatocellular carcinoma.
AU - Xianfen Diao
AU - Ruihao Liu
AU - Boxian Li
AU - Zhaoyang Lu
AU - Zhenhua Gao
AU - Shiting Feng
AU - Jifei Wang
AU - Chuanmiao Xie
AU - Meili Sun
AU - Bingsheng Huang
PY - 2026
DA - 2026/10/10
JO - Quantitative imaging in medicine and surgery
DO - 10.21037/qims-2026-0513
UR - https://europepmc.org/article/MED/42997797
AB - Hepatocellular carcinoma (HCC) is one of the most common malignant tumors in China, and its prognosis is affected by many factors, such as tumor stage, liver function status, and patient health. To formulate treatment plans and evaluate prognosis, clinicians require accurate prediction of early postoperative recurrence of HCC. To address this clinical need, this study proposes a novel deep learning framework, termed MIFF-CLIP-HCC, to predict the early postoperative recurrence of HCC based on multimodal information fusion framework (MIFF) and Contrastive Language-Image Pretraining (CLIP). The MIFF-CLIP-HCC framework comprises two core steps. It first uses a fusion framework with a bidirectional attention mechanism to integrate multimodal information, including arterial phase images, venous phase images, and clinical data from patients with HCC. It then introduces image features extracted by the foundational CLIP model to deeply mine effective information from arterial and venous phase images, which increases the performance of the model. Between 2022 and 2025, a total of 240 early-stage HCC patients from two centers were included. Of these, 180 were from The First Affiliated Hospital of Sun Yat-sen University (Center 1) (including 114 early recurrence cases and 66 non-early recurrence cases, with an 8:2 ratio for training and internal testing), and the remaining 60 were from Sun Yat-sen University Cancer Center (Center 2) (including 35 early recurrence cases and 25 non-early recurrence cases), designated as the external test cohort. The accuracy and robustness of the model were validated on internal and external test sets. The model achieved area under the receiver operating characteristic curve (AUC) values of 0.844 [95% confidence interval (CI): 0.726-0.963] on the internal test set and 0.759 (95% CI: 0.651-0.867) on the external test set, outperforming other fusion frameworks (best baseline AUC: 0.779 on internal and 0.690 on external). The results demonstrate that the proposed MIFF-CLIP-HCC framework, which employs dual optimization strategies of multimodal information fusion and CLIP, significantly improves the accuracy of early postoperative recurrence prediction for HCC, thus demonstrating considerable potential for clinical application.
SN - 2223-4292
SN - 2223-4306
ER -