GAMPoint: Geometry-augmented vision transformer with a multimodal-ready interface for 3D point-cloud segmentation
- 1Department of Artificial Intelligence, School of Artificial Intelligence, Guangzhou Institute of Science and Technology, Guangzhou, China.
- 2South China Normal University Affiliated Huadu School, Guangzhou, China.
- 3Faculty of Science and Technology, University of Macau, Macau, China.
Abstract
Various 3D point cloud semantic segmentation methods have been proposed, yet they remain challenged by limited use of geometric cues and inaccurate predictions around thin structures, object boundaries, and rare classes in cluttered indoor scenes. To address these limitations, we propose GAMPoint, a transformer-based framework for 3D semantic segmentation. Specifically, Enhanced Relative Positional Encoding (EnhancedRPE) augments relative positional modeling with local surface-variation cues to strengthen geometry-aware feature learning. In addition, Conditional Multimodal Gating (CMG) is introduced as an input-side embedding interface. In GAMPoint, XYZ coordinates provide the basic point-cloud geometry, while CMG incorporates available auxiliary attributes, such as RGB colors and surface normals, before the serialized encoder. For RGB-D reconstructed datasets such as ScanNet, XYZ coordinates already contain depth-derived geometric information; raw depth is not used as an additional per-point channel. GAMPoint incorporates EnhancedRPE and CMG into Point Transformer V3 and is evaluated on S3DIS and ScanNet. Experimental results show competitive performance and modest gains over strong point-transformer baselines. Class-wise and repeated-run analyses further indicate category-dependent improvements and consistent gains across matched repeated runs under the reported settings. Code is available at github.com/airmanss/ GAMPoint.
Related
- Research on an elderly low-vision field-aid system based on augmented reality
- CBAT-Net: a unified CNN–BiLSTM–MultiHead transformer architecture for joint Q3/Q8 protein secondary structure prediction
- A novel LLM-based GPT-neo transformer with machine learning architecture for promoters prediction in Agrobacterium
- RNAbpFlow: base pair-augmented SE(3) flow matching for conditional RNA 3D structure generation
- Deep Learning-Augmented Stimulated Raman Imaging for Cell-Type-Specific Metabolic Profiling in Live Neuronal Cocultures.
BibTeX
@article{mu2026gampoint,
title = {{GAMPoint: Geometry-augmented vision transformer with a multimodal-ready interface for 3D point-cloud segmentation}},
author = {Xinyu Mu and Xuan He and Ziqi Huang and Ting Liu},
journal = {PLOS One},
year = {2026},
month = oct,
doi = {10.1371/journal.pone.0360907},
url = {https://doi.org/10.1371/journal.pone.0360907},
note = {CC BY},
}
RIS
TY - JOUR
TI - GAMPoint: Geometry-augmented vision transformer with a multimodal-ready interface for 3D point-cloud segmentation
AU - Xinyu Mu
AU - Xuan He
AU - Ziqi Huang
AU - Ting Liu
PY - 2026
DA - 2026/10/23
JO - PLOS One
DO - 10.1371/journal.pone.0360907
UR - https://doi.org/10.1371/journal.pone.0360907
AB - Various 3D point cloud semantic segmentation methods have been proposed, yet they remain challenged by limited use of geometric cues and inaccurate predictions around thin structures, object boundaries, and rare classes in cluttered indoor scenes. To address these limitations, we propose GAMPoint, a transformer-based framework for 3D semantic segmentation. Specifically, Enhanced Relative Positional Encoding (EnhancedRPE) augments relative positional modeling with local surface-variation cues to strengthen geometry-aware feature learning. In addition, Conditional Multimodal Gating (CMG) is introduced as an input-side embedding interface. In GAMPoint, XYZ coordinates provide the basic point-cloud geometry, while CMG incorporates available auxiliary attributes, such as RGB colors and surface normals, before the serialized encoder. For RGB-D reconstructed datasets such as ScanNet, XYZ coordinates already contain depth-derived geometric information; raw depth is not used as an additional per-point channel. GAMPoint incorporates EnhancedRPE and CMG into Point Transformer V3 and is evaluated on S3DIS and ScanNet. Experimental results show competitive performance and modest gains over strong point-transformer baselines. Class-wise and repeated-run analyses further indicate category-dependent improvements and consistent gains across matched repeated runs under the reported settings. Code is available at github.com/airmanss/ GAMPoint.
SN - 1932-6203
ER -