Multi-criteria prioritized reinforcement learning for autonomous underwater vehicle navigation in 3D multi-node environments
Abstract
Autonomous underwater vehicles (AUVs) are increasingly used as mobile data collectors in underwater wireless sensor networks, where navigation must balance task completion, obstacle avoidance, communication performance, and energy efficiency. This study develops a Twin Delayed Deep Deterministic Policy Gradient framework with Multi-Criteria Prioritized Experience Replay (MC-PER-TD3). The replay priority combines normalized temporal-difference error with a goal-proximity heuristic; energy consumption is incorporated into the reward and data-collection evaluation rather than the replay priority. A soft potential-field reward and a time-window-based waypoint escape-transition mechanism are further used to guide continuous navigation. Across six training seeds and 60 deterministic-policy evaluation episodes per seed, MC-PER-TD3 achieved 100% mission success with zero observed collisions. Relative to uniform-replay TD3, the mean successful path length decreased from 13.188 ± 3.885 m to 10.985 ± 0.436 m and task time from 6.911 ± 1.511 s to 5.654 ± 0.257 s. A standard TD-error-only PER-TD3 baseline achieved 50% mission success, whereas the combined TD-error-plus-goal replay achieved consistent completion across all six training seeds. Additional zero-shot tests under observation noise, current perturbations, fixed and randomized obstacle layouts reveal condition-dependent robustness and clear failure boundaries. In UWSN evaluation, MC-PER-TD3 achieved the highest mean energy efficiency, while TD3 retained the highest throughput, indicating a throughput-energy trade-off rather than universal dominance.
Related
- Reinforcement learning enables autonomous microrobot navigation and intervention in simulated blood capillaries
- Autonomous Droplet Navigation via Model-Based Reinforcement Learning
- Underwater Navigation in Unsteady Flows Using Measurement Histories from a Single Sensing Unit
- A Fermatean fuzzy CRITIC WASPAS framework for conflict detection in autonomous drone navigation
- Hybrid Vision Perception for Low-Cost Autonomous Vehicle Navigation
BibTeX
@article{yan2026multicriteria,
title = {{Multi-criteria prioritized reinforcement learning for autonomous underwater vehicle navigation in 3D multi-node environments}},
author = {Luoheng Yan and Qiongyao Zhang and Zhongmin Huangfu and Dachun Shangguan},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-77136-x},
url = {https://www.nature.com/articles/s41598-026-77136-x},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Multi-criteria prioritized reinforcement learning for autonomous underwater vehicle navigation in 3D multi-node environments
AU - Luoheng Yan
AU - Qiongyao Zhang
AU - Zhongmin Huangfu
AU - Dachun Shangguan
PY - 2026
DA - 2026/10/21
JO - Scientific Reports
DO - 10.1038/s41598-026-77136-x
UR - https://www.nature.com/articles/s41598-026-77136-x
AB - Autonomous underwater vehicles (AUVs) are increasingly used as mobile data collectors in underwater wireless sensor networks, where navigation must balance task completion, obstacle avoidance, communication performance, and energy efficiency. This study develops a Twin Delayed Deep Deterministic Policy Gradient framework with Multi-Criteria Prioritized Experience Replay (MC-PER-TD3). The replay priority combines normalized temporal-difference error with a goal-proximity heuristic; energy consumption is incorporated into the reward and data-collection evaluation rather than the replay priority. A soft potential-field reward and a time-window-based waypoint escape-transition mechanism are further used to guide continuous navigation. Across six training seeds and 60 deterministic-policy evaluation episodes per seed, MC-PER-TD3 achieved 100% mission success with zero observed collisions. Relative to uniform-replay TD3, the mean successful path length decreased from 13.188 ± 3.885 m to 10.985 ± 0.436 m and task time from 6.911 ± 1.511 s to 5.654 ± 0.257 s. A standard TD-error-only PER-TD3 baseline achieved 50% mission success, whereas the combined TD-error-plus-goal replay achieved consistent completion across all six training seeds. Additional zero-shot tests under observation noise, current perturbations, fixed and randomized obstacle layouts reveal condition-dependent robustness and clear failure boundaries. In UWSN evaluation, MC-PER-TD3 achieved the highest mean energy efficiency, while TD3 retained the highest throughput, indicating a throughput-energy trade-off rather than universal dominance.
SN - 2045-2322
ER -