UAMP: Consistent video object segmentation with uncertainty-aware memory propagation
- 1Department of Intelligent Science and Engineering, Xi’an Peihua University, Xi’an, China
- 2Trine Engineering Institute, Shaanxi University of Technology, Hanzhong, China
Abstract
The Segment Anything Model 2 (SAM 2) has emerged as a robust foundational model for video object segmentation. Nevertheless, it exhibits notable limitations in crowded scenarios, particularly those involving fast-moving objects or self-occlusion. Furthermore, its greedy-selection memory architecture is afflicted by “error accumulation,” which collectively impairs its performance in consistent object segmentation tasks. To address these critical drawbacks and enhance the consistency and robustness of SAM 2 in complex video scenarios, an enhanced variant of SAM 2, termed UAMP, is proposed, which integrates memory propagation based on explicit appearance and motion uncertainty modeling, coupled with a dual mechanism encompassing long-term memory updating and short-term memory selection. By fusing uncertainty-aware representations with these dual memory mechanisms, UAMP effectively accommodates dynamic variations in object appearance and motion, further refines the object memory bank, and thereby realizes consistent video object segmentation. Quantitative and qualitative evaluations conducted on diverse benchmark datasets demonstrate that UAMP achieves superior performance, particularly in scenarios involving occlusions and object reappearances. Specifically, UAMP yields consistent performance improvements over state-of-the-art (SOTA) methods across five video object segmentation (VOS) benchmarks, with a maximum enhancement of 5.6 points in the J&F metric. These findings underscore the robustness and effectiveness of the proposed UAMP method in addressing complex tracking scenarios, thus providing a valuable enhancement to SAM 2 for practical video object segmentation applications.
Related
- Deep Learning-Meets-Active Surfaces: A Unified Framework for Multi-Object Segmentation and Shape Analysis in 3D Fluorescence Microscopy
- Retrieval tasks differentially influence memory for object features
- NeuroWeave-3D: Uncertainty-Aware Evidence Completion for Brain Tumor Segmentation with Missing MRI Modalities
- Perturbation-based propagation of bacterial composition uncertainty improves robustness and reproducibility of microbiome differential abundance analysis.
- Object Details Persist in Visual Working Memory Regardless of Category Sufficiency for Search
BibTeX
@article{luo2026uamp,
title = {{UAMP: Consistent video object segmentation with uncertainty-aware memory propagation}},
author = {Yichuang Luo and Fang Wang and Xiaohu Liu},
journal = {PLOS One},
year = {2026},
month = jul,
doi = {10.1371/journal.pone.0353156},
url = {https://doi.org/10.1371/journal.pone.0353156},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - UAMP: Consistent video object segmentation with uncertainty-aware memory propagation
AU - Yichuang Luo
AU - Fang Wang
AU - Xiaohu Liu
PY - 2026
DA - 2026/07/07
JO - PLOS One
DO - 10.1371/journal.pone.0353156
UR - https://doi.org/10.1371/journal.pone.0353156
AB - The Segment Anything Model 2 (SAM 2) has emerged as a robust foundational model for video object segmentation. Nevertheless, it exhibits notable limitations in crowded scenarios, particularly those involving fast-moving objects or self-occlusion. Furthermore, its greedy-selection memory architecture is afflicted by “error accumulation,” which collectively impairs its performance in consistent object segmentation tasks. To address these critical drawbacks and enhance the consistency and robustness of SAM 2 in complex video scenarios, an enhanced variant of SAM 2, termed UAMP, is proposed, which integrates memory propagation based on explicit appearance and motion uncertainty modeling, coupled with a dual mechanism encompassing long-term memory updating and short-term memory selection. By fusing uncertainty-aware representations with these dual memory mechanisms, UAMP effectively accommodates dynamic variations in object appearance and motion, further refines the object memory bank, and thereby realizes consistent video object segmentation. Quantitative and qualitative evaluations conducted on diverse benchmark datasets demonstrate that UAMP achieves superior performance, particularly in scenarios involving occlusions and object reappearances. Specifically, UAMP yields consistent performance improvements over state-of-the-art (SOTA) methods across five video object segmentation (VOS) benchmarks, with a maximum enhancement of 5.6 points in the J&F metric. These findings underscore the robustness and effectiveness of the proposed UAMP method in addressing complex tracking scenarios, thus providing a valuable enhancement to SAM 2 for practical video object segmentation applications.
SN - 1932-6203
ER -