Robust disturbance-aware actor–critic reinforcement learning for multi-DOF robotic manipulators
- 1Faculty of Electrical and Electronic Engineering, Hung Yen University of Technology and Education, Vietnam
Abstract
Building on recent insights that augmenting reinforcement‐learning policies with disturbance estimates improves robustness and sim-to-real transfer, this paper proposes a disturbance-aware actor–critic RL framework for high‐precision robotic manipulators. We derive the dynamics of manipulators ranging from two to six degrees of freedom and design a nonlinear disturbance observer that provides real-time estimates of lumped uncertainties. Unlike conventional feedforward-only designs, the proposed DACRL framework deeply integrates DOB information into the actor-critic’s learning state, cost function, and update laws to explicitly compensate for unknown dynamics and disturbances. A Lyapunov-based analysis proves that tracking errors, observer errors and neural-network weight errors remain uniformly ultimately bounded. Extensive simulations show that the proposed controller achieves sub-degree tracking errors across multiple DOF cases, with the 2-DOF example achieving steady-state errors below ±0.02 rad (Joint 1) and ±0.05 rad (Joint 2) and delivering faster convergence and smoother torques than conventional RL and DOB baselines. The disturbance-aware controller generalizes across trajectories and payloads, offering improved robustness while retaining learning flexibility. While the current study focuses on performance and robustness, future work could explore the integration of disturbance-observer-based control barrier functions to formally address safety constraints during the learning process. Simulation results suggest that the disturbance-aware controller can improve robustness while retaining learning flexibility. Real-platform validation remains an important direction for future work.
Related
- Meta reinforcement learning for robust quadruped locomotion under joint failures
- Robust and Quality-of-Life-Aware Treatment Protocols in NSCLC using Deep Reinforcement Learning
- Finite-time robust motion control of intelligent vehicles based on neural approximation and disturbance estimation
- DINOCT: robust skin surface localization for robotic noncontact dynamic optical coherence elastography
- Trajectory tracking control of unmanned aerial vehicle with disturbance observer and robust adaptive neural dynamic surface
BibTeX
@article{nguyen2026robust,
title = {{Robust disturbance-aware actor–critic reinforcement learning for multi-DOF robotic manipulators}},
author = {Viet Ngu Nguyen and Thi Minh Tam Le and Duc Hung Pham and Huu Cuong Nguyen},
journal = {PLOS One},
year = {2026},
month = oct,
doi = {10.1371/journal.pone.0354740},
url = {https://doi.org/10.1371/journal.pone.0354740},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Robust disturbance-aware actor–critic reinforcement learning for multi-DOF robotic manipulators
AU - Viet Ngu Nguyen
AU - Thi Minh Tam Le
AU - Duc Hung Pham
AU - Huu Cuong Nguyen
PY - 2026
DA - 2026/10/06
JO - PLOS One
DO - 10.1371/journal.pone.0354740
UR - https://doi.org/10.1371/journal.pone.0354740
AB - Building on recent insights that augmenting reinforcement‐learning policies with disturbance estimates improves robustness and sim-to-real transfer, this paper proposes a disturbance-aware actor–critic RL framework for high‐precision robotic manipulators. We derive the dynamics of manipulators ranging from two to six degrees of freedom and design a nonlinear disturbance observer that provides real-time estimates of lumped uncertainties. Unlike conventional feedforward-only designs, the proposed DACRL framework deeply integrates DOB information into the actor-critic’s learning state, cost function, and update laws to explicitly compensate for unknown dynamics and disturbances. A Lyapunov-based analysis proves that tracking errors, observer errors and neural-network weight errors remain uniformly ultimately bounded. Extensive simulations show that the proposed controller achieves sub-degree tracking errors across multiple DOF cases, with the 2-DOF example achieving steady-state errors below ±0.02 rad (Joint 1) and ±0.05 rad (Joint 2) and delivering faster convergence and smoother torques than conventional RL and DOB baselines. The disturbance-aware controller generalizes across trajectories and payloads, offering improved robustness while retaining learning flexibility. While the current study focuses on performance and robustness, future work could explore the integration of disturbance-observer-based control barrier functions to formally address safety constraints during the learning process. Simulation results suggest that the disturbance-aware controller can improve robustness while retaining learning flexibility. Real-platform validation remains an important direction for future work.
SN - 1932-6203
ER -