Explainable Multimodal Prognostic Modeling Uncovers New Clinically Relevant Pathological Features in a Multi-Institutional Pancreatic Neuroendocrine Tumor Cohort
- 1Johns Hopkins University School of Medicine
Abstract
Although many patients with a well-differentiated pancreatic neuroendocrine tumor (PanNET) are cured after surgical resection, it is hard to predict outcome in these patients. Current prognostic assessment relies largely on Ki-67 proliferation index, tumor stage, and established adverse pathologic features, such as vascular and perineural invasion. However, individually, these variables incompletely explain why patients with similar clinicopathologic profiles sometimes experience markedly different outcomes. To address this challenge, we assembled three independent cohorts with a surgically resected PanNET. In total, we included 1,438 patients with approximately 30,000 histology images, and approximately 12,000 person-years of follow-up. We separately modeled structured clinicopathologic information with Cox regression and tumor-centered pathology images with deep learning on an internal cohort treated at Johns Hopkins Hospital and performed multimodal continuous-time survival modeling to reveal which clinical and pathologic features are most important in predicting patient outcome. Across Cox models of censored time to recurrence-free survival, a text-based clinicopathological model that incorporated known prognosticators achieved a C-index of approximately 0.87, the image-based histology model approximately 0.80, and decision-level late multimodal fusion approximately 0.88 on the internal test set. The clinical model demonstrated strong prognostic discrimination using established risk factors, mainly relying on the TNM stage and Ki-67 index. Primary tumor histology retained substantial prognostic signal, with analysis restricted to tumor-centered regions to reduce reliance on unrelated tissues that might indirectly indicate disease extent. Interpretation of high-attention image tiles revealed regions of importance in the primary tumor, including recognized adverse morphologic features such as the leading edge of the tumor and foci of vascular or perineural invasion. Unexpectedly, intratumoral and peritumoral stroma accounted for nearly 50% of the model's high-attention regions, suggesting that the stroma may yield equal prognostic information to neoplastic cells in recurrence-prediction models. To test generalizability, we further evaluated the trained models in independent cohorts from two institutions with different demographics, where discrimination varied by site (mean late-fusion C-indices of 0.864 and 0.723). Together with the improved discrimination achieved through late fusion, our findings suggest that current clinicopathologic variables can be modeled to form reliable prognosticators, and that routine hematoxylin and eosin (H&E) stained slides of the primary tumor contain additional prognostically important information not fully captured by conventional clinicopathologic assessment, such that together, the two sources provide complementary information.
Related
- Prognostic factors and survival outcomes of multimodal treatment in sinonasal neuroendocrine tumors in a population based SEER analysis
- MRI-Based Radiomics Model for Pretreatment Prediction of Pathological Lymph Node Metastasis in Neuroendocrine Carcinoma of the Cervix: Dual-Center Development and External Validation.
- From Splicing to Survival: Understanding the Molecular Basis of Prognostic Differences in advanced Small Intestine Neuroendocrine Tumors
- Treatment-Free Interval as a Prognostic Factor for Survival Outcomes in Patients with Advanced or Recurrent Endometrial Cancer Treated with Pembrolizumab Plus Lenvatinib: A Multi-Institutional Real-World Cohort Study
- Explainable AI framework for Alzheimer’s disease prediction using multimodal data
BibTeX
@misc{sivasubramanian2026explainable,
title = {{Explainable Multimodal Prognostic Modeling Uncovers New Clinically Relevant Pathological Features in a Multi-Institutional Pancreatic Neuroendocrine Tumor Cohort}},
author = {A. Sivasubramanian and M. Salimian and J. R. Vennapusa and L. Dequiedt and Y. Shen and V. Matos Romero and Y. Zhu and U.-A. Ebong and Z. Gaillard and I. Reucroft and H.-N. Yun and S.-M. Hong and J. Shin and A. Kasajima and G. Klöppel and J. He and R. H. Hruban and A. L. Kiemen},
howpublished = {bioRxiv},
year = {2026},
month = nov,
doi = {10.64898/2026.11.02.763764},
url = {https://www.biorxiv.org/content/10.64898/2026.11.02.763764v1},
note = {CC BY-NC-ND},
}
RIS
TY - UNPB
TI - Explainable Multimodal Prognostic Modeling Uncovers New Clinically Relevant Pathological Features in a Multi-Institutional Pancreatic Neuroendocrine Tumor Cohort
AU - A. Sivasubramanian
AU - M. Salimian
AU - J. R. Vennapusa
AU - L. Dequiedt
AU - Y. Shen
AU - V. Matos Romero
AU - Y. Zhu
AU - U.-A. Ebong
AU - Z. Gaillard
AU - I. Reucroft
AU - H.-N. Yun
AU - S.-M. Hong
AU - J. Shin
AU - A. Kasajima
AU - G. Klöppel
AU - J. He
AU - R. H. Hruban
AU - A. L. Kiemen
PY - 2026
DA - 2026/11/02
PB - bioRxiv
DO - 10.64898/2026.11.02.763764
UR - https://www.biorxiv.org/content/10.64898/2026.11.02.763764v1
AB - Although many patients with a well-differentiated pancreatic neuroendocrine tumor (PanNET) are cured after surgical resection, it is hard to predict outcome in these patients. Current prognostic assessment relies largely on Ki-67 proliferation index, tumor stage, and established adverse pathologic features, such as vascular and perineural invasion. However, individually, these variables incompletely explain why patients with similar clinicopathologic profiles sometimes experience markedly different outcomes. To address this challenge, we assembled three independent cohorts with a surgically resected PanNET. In total, we included 1,438 patients with approximately 30,000 histology images, and approximately 12,000 person-years of follow-up. We separately modeled structured clinicopathologic information with Cox regression and tumor-centered pathology images with deep learning on an internal cohort treated at Johns Hopkins Hospital and performed multimodal continuous-time survival modeling to reveal which clinical and pathologic features are most important in predicting patient outcome. Across Cox models of censored time to recurrence-free survival, a text-based clinicopathological model that incorporated known prognosticators achieved a C-index of approximately 0.87, the image-based histology model approximately 0.80, and decision-level late multimodal fusion approximately 0.88 on the internal test set. The clinical model demonstrated strong prognostic discrimination using established risk factors, mainly relying on the TNM stage and Ki-67 index. Primary tumor histology retained substantial prognostic signal, with analysis restricted to tumor-centered regions to reduce reliance on unrelated tissues that might indirectly indicate disease extent. Interpretation of high-attention image tiles revealed regions of importance in the primary tumor, including recognized adverse morphologic features such as the leading edge of the tumor and foci of vascular or perineural invasion. Unexpectedly, intratumoral and peritumoral stroma accounted for nearly 50% of the model's high-attention regions, suggesting that the stroma may yield equal prognostic information to neoplastic cells in recurrence-prediction models. To test generalizability, we further evaluated the trained models in independent cohorts from two institutions with different demographics, where discrimination varied by site (mean late-fusion C-indices of 0.864 and 0.723). Together with the improved discrimination achieved through late fusion, our findings suggest that current clinicopathologic variables can be modeled to form reliable prognosticators, and that routine hematoxylin and eosin (H&E) stained slides of the primary tumor contain additional prognostically important information not fully captured by conventional clinicopathologic assessment, such that together, the two sources provide complementary information.
ER -