Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems
Abstract
Large-scale industrial robot fleets share constrained physical infrastructure, making vehicle travel times dependent on safety separation, intersection access, downstream blocking, and station contention. We study this problem in overhead hoist transport (OHT) systems, a representative ceiling-mounted material-handling system used in semiconductor fabs. Static shortest-path routing cannot account for these time-varying traffic costs, whereas tabular Q-routing adapts online but learns each destination--node--action value independently, limiting information sharing across sparsely visited routing contexts and making startup behavior sensitive to inaccurate value estimates. We propose Neural Double Q-routing, which replaces destination-indexed tables with a shared state--action value network. The network is warm-started through return-to-go regression on mixed simulator-generated routing trajectories and then refined online using Double-Q updates, local congestion correction, and event-stratified structured replay. Across nine matched fleet-size--arrival-rate settings with 100, 150, and 200 OHTs, the proposed framework reduces mean completion time relative to tabular Double Q-routing by 0.8%--8.8%. It achieves the lowest mean completion time among all compared methods in the six 150- and 200-OHT settings, whereas Dijkstra remains best in the three 100-OHT settings. Completed-task counts remain within 1% of tabular Double Q-routing in eight of nine settings, and 95th-percentile completion time decreases in eight settings. In two matched startup scenarios, offline initialization increases the number of completed tasks by up to 23% and reduces tail completion time by up to 15%.
Related
- On the classification of polarized surfaces with weakly Ulrich or initialized aCM tangent bundles
- Who Drives the System? Classifying Mean-Field Particle Systems from Trajectory Data
- Chapter 2: Geometry of the Fitness Surface and Trajectory Dynamics of Replicator Systems
- Optimal Community Recovery by Spectrally Initialized Variational EM in General Stochastic Block Models
- Control Laguerre Tessellation: Semi-discrete Optimal Transport Over Control Systems
BibTeX
@misc{gu2026trajectoryinitialized,
title = {{Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems}},
author = {Cheng Gu and Qiusheng Zhao and Anbang Liu and Shaochong Lin and Max Z. J. Shen},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.30512},
eprint = {2608.30512},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.30512},
}
RIS
TY - UNPB
TI - Trajectory-Initialized Neural Double Q-Routing for Large-Scale Overhead Hoist Transport Systems
AU - Cheng Gu
AU - Qiusheng Zhao
AU - Anbang Liu
AU - Shaochong Lin
AU - Max Z. J. Shen
PY - 2026
DA - 2026/08/31
PB - arXiv
DO - 10.48550/arxiv.2608.30512
UR - https://arxiv.org/abs/2608.30512
AB - Large-scale industrial robot fleets share constrained physical infrastructure, making vehicle travel times dependent on safety separation, intersection access, downstream blocking, and station contention. We study this problem in overhead hoist transport (OHT) systems, a representative ceiling-mounted material-handling system used in semiconductor fabs. Static shortest-path routing cannot account for these time-varying traffic costs, whereas tabular Q-routing adapts online but learns each destination--node--action value independently, limiting information sharing across sparsely visited routing contexts and making startup behavior sensitive to inaccurate value estimates. We propose Neural Double Q-routing, which replaces destination-indexed tables with a shared state--action value network. The network is warm-started through return-to-go regression on mixed simulator-generated routing trajectories and then refined online using Double-Q updates, local congestion correction, and event-stratified structured replay. Across nine matched fleet-size--arrival-rate settings with 100, 150, and 200 OHTs, the proposed framework reduces mean completion time relative to tabular Double Q-routing by 0.8%--8.8%. It achieves the lowest mean completion time among all compared methods in the six 150- and 200-OHT settings, whereas Dijkstra remains best in the three 100-OHT settings. Completed-task counts remain within 1% of tabular Double Q-routing in eight of nine settings, and 95th-percentile completion time decreases in eight settings. In two matched startup scenarios, offline initialization increases the number of completed tasks by up to 23% and reduces tail completion time by up to 15%.
ER -