Realistic Performance Estimation in Small-Cohort Deep Learning and Machine Learning: A Leakage-Controlled Codebase Applied to Meningioma Response Prediction After Gamma Knife Radiosurgery
4 affiliations
- 1Centro Gamma Knife Dominicano, Centro de Diagnóstico y Medicina Avanzada y de Conferencias Médicas y Telemedicina, Plaza de la Salud, Santo Domingo 10514, Dominican Republic
- 2Department of Radiology, CEDIMAT, Plaza de la Salud, Santo Domingo 10511, Dominican Republic
- 3Department of Basic and Environmental Science, Instituto Tecnológico de Santo Domingo (INTEC), Santo Domingo 10602, Dominican Republic
- 4Department of Experimental Oncology, Institute for Oncology & Radiology of Serbia, 11000 Belgrade, Serbia
Abstract
Background/Objectives: Deep learning may support the prediction of meningioma response after Gamma Knife radiosurgery (GKRS), but small clinical cohorts increase the risk of overfitting and unstable performance. We developed leakage-controlled code adjusted for small clinical datasets. Methods: Pretreatment contrast-enhanced T1 MRI from 190 patients was analysed using one tumour-centred axial slice per patient. Response was defined as a tumour volume decrease ≥ 30% per log10 (months). ResNet-10t, ResNet-18, ResNet-50 and DenseNet-121 with ImageNet or RadImageNet pretraining were evaluated in three workflows: (1) frozen backbone CNN, head-only training; (2) logistic regression or naive Bayes classification of features extracted from frozen CNNs; and (3) restricted CNN fine-tuning. Five partially overlapping chronological development/test splits were assessed, with all data-dependent fitting, feature selection and model optimisation restricted to development data. Results: The smallest RadImageNet ResNet-10t CNN performed best when fine-tuned (peak AUCs 0.953 and 1.000) but showed the widest split-to-split variation (range 0.738–1.000). When RadImageNet features extracted from ResNet-10t were classified by naive Bayes and logistic regression, they reached lower median and peak AUCs but within narrower ranges. Predictive performance did not consistently depend on CNN size. Conclusions: To our knowledge, this is the first deep learning study predicting volumetric meningioma response after GKRS. We provide openly available, ready-to-use code for three major workflows in small clinical datasets, with safeguards against information leakage. The benefit of fine-tuning and of RadImageNet pretraining depended on architecture. Robust small-cohort modelling requires attention to performance consistency across chronological splits, not only to the highest AUC in a single split.
Related
- Multiple sclerosis relapse following gamma knife radiosurgery for meningioma in a patient treated with natalizumab: case report
- Comparison of endoplasmic reticulum chaperone gene expression and dosimetric findings before and after gamma knife radiosurgery in meningioma
- Uncertainty-aware estimation of compressive strength for fly ash concrete with enhanced probabilistic prediction performance by incorporating the associated mortar strength
- MRI-based preoperative WHO grade prediction in meningioma: development and comparison of deep learning, radiomics, clinical, and combined models in multicenter data.
- Multiparametric MRI radiomics and clinical features for preoperative prediction of high-grade meningioma: a two-centre retrospective study
BibTeX
@article{speckter2026realistic,
title = {{Realistic Performance Estimation in Small-Cohort Deep Learning and Machine Learning: A Leakage-Controlled Codebase Applied to Meningioma Response Prediction After Gamma Knife Radiosurgery}},
author = {Herwin Speckter and Marko Radulovic and Ivan Gonzalez and Giancarlo Hernandez and Jose Bido and Diones Rivera and Luis Suazo and Santiago Valenzuela and Ismael Peralta and Jeffrey Paulino and Teuddis Bernard and Issael Ramirez and Peter Stoeter and Velicko Vranes},
journal = {Cancers},
year = {2026},
month = nov,
doi = {10.3390/cancers18213590},
url = {https://www.mdpi.com/2072-6694/18/21/3590},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Realistic Performance Estimation in Small-Cohort Deep Learning and Machine Learning: A Leakage-Controlled Codebase Applied to Meningioma Response Prediction After Gamma Knife Radiosurgery
AU - Herwin Speckter
AU - Marko Radulovic
AU - Ivan Gonzalez
AU - Giancarlo Hernandez
AU - Jose Bido
AU - Diones Rivera
AU - Luis Suazo
AU - Santiago Valenzuela
AU - Ismael Peralta
AU - Jeffrey Paulino
AU - Teuddis Bernard
AU - Issael Ramirez
AU - Peter Stoeter
AU - Velicko Vranes
PY - 2026
DA - 2026/11/09
JO - Cancers
DO - 10.3390/cancers18213590
UR - https://www.mdpi.com/2072-6694/18/21/3590
AB - Background/Objectives: Deep learning may support the prediction of meningioma response after Gamma Knife radiosurgery (GKRS), but small clinical cohorts increase the risk of overfitting and unstable performance. We developed leakage-controlled code adjusted for small clinical datasets. Methods: Pretreatment contrast-enhanced T1 MRI from 190 patients was analysed using one tumour-centred axial slice per patient. Response was defined as a tumour volume decrease ≥ 30% per log10 (months). ResNet-10t, ResNet-18, ResNet-50 and DenseNet-121 with ImageNet or RadImageNet pretraining were evaluated in three workflows: (1) frozen backbone CNN, head-only training; (2) logistic regression or naive Bayes classification of features extracted from frozen CNNs; and (3) restricted CNN fine-tuning. Five partially overlapping chronological development/test splits were assessed, with all data-dependent fitting, feature selection and model optimisation restricted to development data. Results: The smallest RadImageNet ResNet-10t CNN performed best when fine-tuned (peak AUCs 0.953 and 1.000) but showed the widest split-to-split variation (range 0.738–1.000). When RadImageNet features extracted from ResNet-10t were classified by naive Bayes and logistic regression, they reached lower median and peak AUCs but within narrower ranges. Predictive performance did not consistently depend on CNN size. Conclusions: To our knowledge, this is the first deep learning study predicting volumetric meningioma response after GKRS. We provide openly available, ready-to-use code for three major workflows in small clinical datasets, with safeguards against information leakage. The benefit of fine-tuning and of RadImageNet pretraining depended on architecture. Robust small-cohort modelling requires attention to performance consistency across chronological splits, not only to the highest AUC in a single split.
SN - 2072-6694
ER -