UniMedSeg: Unified Multi-Modal Medical Image Segmentation via Uncertainty Guided Encoding and Frequency-Spatial Dual-Prompt Decoding
- 1City University of Macau
Abstract
Text-driven medical image segmentation demands unified models based on multi-modal learning, capable of processing diverse imaging modalities while providing spatially reliable prediction estimates for clinical practice. Nevertheless, existing vision-language model solutions suffer from poor domain generalization and exhibit limited capability for uncertainty estimation to model the cumulative unreliability of cross-modal alignment across network layers. In this paper, we propose UniMedSeg, a unified vision-language framework for multi-modal medical image segmentation built upon frozen pre-trained vision-language backbones. Our framework incorporates modality-aware adaptation, cross-layer uncertainty-guided encoding, and a frequency-spatial dual-prompt decoder, wherein the progressive uncertainty propagation mechanism of the encoder suppresses misleading cross-modal features, whereas the decoder disentangles anatomical structure and boundary details in the frequency domain under dual textual prompts. Auxiliary losses further sharpen boundary representations and calibrate pixel-wise uncertainty maps. Extensive experiments demonstrate that UniMedSeg preserves competitive in-domain segmentation performance, improves domain generalization, and yields clinically useful spatial risk estimation.
Related
- TiLCA-Net: nuclei segmentation and classification in pathological images via task-specialized attention and efficient decoding
- Encoding and decoding of acupuncture-evoked electrical signals and their value in acupuncture–brain-computer interface-integrated closed-loop regulation
- UGB-SegNet: Uncertainty-Guided Multi-Scale Attention and Boundary-Aware Learning for Breast Ultrasound Lesion Segmentation
- Swin-WaveNet: frequency-guided boundary restoration for robust skin lesion segmentation.
- Fuzzy-logic enhanced internet of things model for uncertainty-adaptive medical image segmentation
BibTeX
@misc{zhang2026unimedseg,
title = {{UniMedSeg: Unified Multi-Modal Medical Image Segmentation via Uncertainty Guided Encoding and Frequency-Spatial Dual-Prompt Decoding}},
author = {J. Zhang and S. t. Zheng and X. f. Liu and W. Zhang and J. Gu},
howpublished = {bioRxiv},
year = {2026},
month = oct,
doi = {10.64898/2026.10.03.755900},
url = {https://www.biorxiv.org/content/10.64898/2026.10.03.755900v1},
note = {CC BY-NC},
}
RIS
TY - UNPB
TI - UniMedSeg: Unified Multi-Modal Medical Image Segmentation via Uncertainty Guided Encoding and Frequency-Spatial Dual-Prompt Decoding
AU - J. Zhang
AU - S. t. Zheng
AU - X. f. Liu
AU - W. Zhang
AU - J. Gu
PY - 2026
DA - 2026/10/03
PB - bioRxiv
DO - 10.64898/2026.10.03.755900
UR - https://www.biorxiv.org/content/10.64898/2026.10.03.755900v1
AB - Text-driven medical image segmentation demands unified models based on multi-modal learning, capable of processing diverse imaging modalities while providing spatially reliable prediction estimates for clinical practice. Nevertheless, existing vision-language model solutions suffer from poor domain generalization and exhibit limited capability for uncertainty estimation to model the cumulative unreliability of cross-modal alignment across network layers. In this paper, we propose UniMedSeg, a unified vision-language framework for multi-modal medical image segmentation built upon frozen pre-trained vision-language backbones. Our framework incorporates modality-aware adaptation, cross-layer uncertainty-guided encoding, and a frequency-spatial dual-prompt decoder, wherein the progressive uncertainty propagation mechanism of the encoder suppresses misleading cross-modal features, whereas the decoder disentangles anatomical structure and boundary details in the frequency domain under dual textual prompts. Auxiliary losses further sharpen boundary representations and calibrate pixel-wise uncertainty maps. Extensive experiments demonstrate that UniMedSeg preserves competitive in-domain segmentation performance, improves domain generalization, and yields clinically useful spatial risk estimation.
ER -