AI-based single-shot structured-light depth reconstruction for real-time laparoscopic surgical guidance
Abstract
Significance. Accurate intraoperative depth perception is important for autonomous and semi-autonomous robotic laparoscopic surgery. Conventional fringe projection profilometry can achieve millimeter-scale accuracy but often requires multi-shot acquisition, digital-micromirror-device projection, and projector-camera synchronization, complicating integration into compact laparoscopic systems. Aim. To develop a synchronization-free, single-shot depth-sensing platform using a passive LED-illuminated binary mask and a VQ-VAE prior with a custom U-Net depth head. Approach. A compact projection module was coupled to one channel of a dual-channel laparoscope, while the second channel imaged the fringe-illuminated target. A Zivid 3D camera acquired reference depth for 722 paired phantom images. Zivid depth maps were reprojected into the SSLE image frame for supervised training and evaluation. The VQ-VAE encoded each input into a discrete latent representation, and a latent-space U-Net predicted depth without a separate mask-prediction branch. Results. Using a fixed train/validation/test split, the proposed model achieved an MAE of 3.70 mm, AbsRel of 0.0326, delta=1.1 accuracy of 0.962, and delta=1.1^2 accuracy of 0.970. It achieved lower MAE than the dual U-Net MaskNet + DepthNet baseline and outperformed off-the-shelf monocular depth models in MAE, AbsRel, and threshold accuracy. The pipeline operated at 26.0 Hz over 301 consecutive frames on an NVIDIA A100 GPU. Conclusions. The LED-illuminated binary-pattern platform with latent-space depth reconstruction enables synchronization-free, video-rate endoscopic depth estimation. Results demonstrate Zivid-referenced phantom reconstruction without an explicit segmentation stage, while emphasizing the importance of dataset size and SSLE-Zivid calibration accuracy.
Related
- Learning Transferable Self-Supervised Priors for Super-Resolution Reconstruction in Structured Illumination Microscopy
- Reconstruction of the superstatistical temperature distribution from single-particle kinetic energies
- Three-dimensional reconstruction of weakly scattering objects from single-defocus 4D-STEM data
- Role of Spatial Coherence in Single-Shot Lensless Image Reconstruction
- Single acquisition reconstruction of nonlinear susceptibility and Raman tensors at the diffraction limit
BibTeX
@misc{rodgers2026aibased,
title = {{AI-based single-shot structured-light depth reconstruction for real-time laparoscopic surgical guidance}},
author = {Wayne Wonseok Rodgers and Xiangyi Le and Seonghoon Jang and Shuwen Wei and Justin Opfermann and Michael Kam and Axel Krieger and Jin U. Kang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.05109},
eprint = {2608.05109},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.05109},
}
RIS
TY - UNPB
TI - AI-based single-shot structured-light depth reconstruction for real-time laparoscopic surgical guidance
AU - Wayne Wonseok Rodgers
AU - Xiangyi Le
AU - Seonghoon Jang
AU - Shuwen Wei
AU - Justin Opfermann
AU - Michael Kam
AU - Axel Krieger
AU - Jin U. Kang
PY - 2026
DA - 2026/09/04
PB - arXiv
DO - 10.48550/arxiv.2608.05109
UR - https://arxiv.org/abs/2608.05109
AB - Significance. Accurate intraoperative depth perception is important for autonomous and semi-autonomous robotic laparoscopic surgery. Conventional fringe projection profilometry can achieve millimeter-scale accuracy but often requires multi-shot acquisition, digital-micromirror-device projection, and projector-camera synchronization, complicating integration into compact laparoscopic systems. Aim. To develop a synchronization-free, single-shot depth-sensing platform using a passive LED-illuminated binary mask and a VQ-VAE prior with a custom U-Net depth head. Approach. A compact projection module was coupled to one channel of a dual-channel laparoscope, while the second channel imaged the fringe-illuminated target. A Zivid 3D camera acquired reference depth for 722 paired phantom images. Zivid depth maps were reprojected into the SSLE image frame for supervised training and evaluation. The VQ-VAE encoded each input into a discrete latent representation, and a latent-space U-Net predicted depth without a separate mask-prediction branch. Results. Using a fixed train/validation/test split, the proposed model achieved an MAE of 3.70 mm, AbsRel of 0.0326, delta=1.1 accuracy of 0.962, and delta=1.1^2 accuracy of 0.970. It achieved lower MAE than the dual U-Net MaskNet + DepthNet baseline and outperformed off-the-shelf monocular depth models in MAE, AbsRel, and threshold accuracy. The pipeline operated at 26.0 Hz over 301 consecutive frames on an NVIDIA A100 GPU. Conclusions. The LED-illuminated binary-pattern platform with latent-space depth reconstruction enables synchronization-free, video-rate endoscopic depth estimation. Results demonstrate Zivid-referenced phantom reconstruction without an explicit segmentation stage, while emphasizing the importance of dataset size and SSLE-Zivid calibration accuracy.
ER -