Convolutional, State-Space and Transformer Liver-Tumor Segmentation Models Trained on Hepatocellular Carcinoma Fail on the Same External Cases: Lesion Size Outweighs Architecture in Cross-Dataset Generalization
- 1Department of Biostatistics and Medical Informatics, Faculty of Medicine, Adıyaman University, 02040 Adıyaman, Türkiye
- 2Department of Biostatistics and Medical Informatics, Faculty of Medicine, İnönü University, 44280 Malatya, Türkiye
- 3Department of Surgery and Liver Transplantation, Faculty of Medicine, İnönü University, 44280 Malatya, Türkiye
Abstract
Background/Objectives: Architectural innovation is widely assumed to improve the cross-institutional generalization of tumor segmentation models, yet this is rarely tested. We compared three architectural families under an identical, leak-safe protocol in hepatocellular carcinoma (HCC) segmentation. Methods: Convolutional (nnU-Net ResEnc-M), state-space (U-Mamba Bot) and transformer (SwinUNETR) models were trained on HCC-TACE-Seg (n = 103) with identical patient-disjoint five-fold splits, then applied without adaptation to 3D-IRCADb (n = 15) and LiTS (n = 118). We assessed discrimination, case-level failure concordance, lesion-volume dependence, calibration and selective prediction. Results: Internal performance differed (Dice 0.685, 0.662, 0.479; Friedman p = 0.022) but external performance was low and largely convergent: the convolutional and state-space models were formally equivalent on both external cohorts (pre-specified margin ±0.05 Dice), whereas the transformer was inferior on LiTS (0.222; p < 0.0001) and inconclusive on 3D-IRCADb. Case-level Dice was highly concordant across all pairings (Spearman ρ = 0.84–0.96; 0.67–0.81 on LiTS after adjustment for lesion volume), the same cases failing across architectures. Lesion volume predicted performance in all six architecture × cohort combinations; lesions < 5 mL failed completely in 61–76% of cases and formed 1.9% of the development cohort against 31.4% of LiTS. Neither liver-mask constraint nor ensembling recovered performance. Deferring the 30% least-confident LiTS cases raised Dice to 0.516 (0.450 when only empty predictions were removed). Conclusions: Under a single-source, TACE-treated HCC training regime, cross-dataset failure of liver-tumor segmentation was governed primarily by lesion size rather than by architectural family; confidence-based selective prediction offers a deployable mitigation.
Related
- Deep Learning–driven Segmentation of Breast Cancer MR Imaging Using a Hybrid ResNet–Transformer Architecture and Cross-dataset Validation
- MedFuse: dual-stream fusion of convolutional and vision transformer-based features for enhanced medical image classification
- Interpretable attention-deficit hyperactivity disorder diagnosis from functional magnetic resonance imaging via hybrid convolutional neural network–transformer learning and multi-level graph-based comprehensibility
- Comparative evaluation of convolutional and transformer-based object detection models for brain tumor localization in MRI
- An attention-guided dense convolutional network for automated diabetic foot ulcer segmentation
BibTeX
@article{kutlu2026convolutional,
title = {{Convolutional, State-Space and Transformer Liver-Tumor Segmentation Models Trained on Hepatocellular Carcinoma Fail on the Same External Cases: Lesion Size Outweighs Architecture in Cross-Dataset Generalization}},
author = {Hüseyin Kutlu and Cemil Çolak and Sami Akbulut},
journal = {Cancers},
year = {2026},
month = oct,
doi = {10.3390/cancers18203362},
url = {https://www.mdpi.com/2072-6694/18/20/3362},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Convolutional, State-Space and Transformer Liver-Tumor Segmentation Models Trained on Hepatocellular Carcinoma Fail on the Same External Cases: Lesion Size Outweighs Architecture in Cross-Dataset Generalization
AU - Hüseyin Kutlu
AU - Cemil Çolak
AU - Sami Akbulut
PY - 2026
DA - 2026/10/01
JO - Cancers
DO - 10.3390/cancers18203362
UR - https://www.mdpi.com/2072-6694/18/20/3362
AB - Background/Objectives: Architectural innovation is widely assumed to improve the cross-institutional generalization of tumor segmentation models, yet this is rarely tested. We compared three architectural families under an identical, leak-safe protocol in hepatocellular carcinoma (HCC) segmentation. Methods: Convolutional (nnU-Net ResEnc-M), state-space (U-Mamba Bot) and transformer (SwinUNETR) models were trained on HCC-TACE-Seg (n = 103) with identical patient-disjoint five-fold splits, then applied without adaptation to 3D-IRCADb (n = 15) and LiTS (n = 118). We assessed discrimination, case-level failure concordance, lesion-volume dependence, calibration and selective prediction. Results: Internal performance differed (Dice 0.685, 0.662, 0.479; Friedman p = 0.022) but external performance was low and largely convergent: the convolutional and state-space models were formally equivalent on both external cohorts (pre-specified margin ±0.05 Dice), whereas the transformer was inferior on LiTS (0.222; p < 0.0001) and inconclusive on 3D-IRCADb. Case-level Dice was highly concordant across all pairings (Spearman ρ = 0.84–0.96; 0.67–0.81 on LiTS after adjustment for lesion volume), the same cases failing across architectures. Lesion volume predicted performance in all six architecture × cohort combinations; lesions < 5 mL failed completely in 61–76% of cases and formed 1.9% of the development cohort against 31.4% of LiTS. Neither liver-mask constraint nor ensembling recovered performance. Deferring the 30% least-confident LiTS cases raised Dice to 0.516 (0.450 when only empty predictions were removed). Conclusions: Under a single-source, TACE-treated HCC training regime, cross-dataset failure of liver-tumor segmentation was governed primarily by lesion size rather than by architectural family; confidence-based selective prediction offers a deployable mitigation.
SN - 2072-6694
ER -