Characterization of a fixed reinforcement learning policy for aerial robot with suspended payload under variable flight conditions
Abstract
Flights with suspended payloads are particularly challenging because of their coupled dynamics, which lead to instability and increased sensitivity to disturbances. Although reinforcement learning (RL) has successfully achieved controller performance, the generalization and robustness of a single policy remain significant areas of investigation. In this study, we characterized the performance and robustness of a single RL policy for an aerial robot with different trajectory profiles, including a smooth, feasible lemniscate curve and a sharp-turning, infeasible pentagram, under varying velocity references (0.5 m s −1 and 1.0 m s −1 ) and crosswind disturbances (1.0 m s −1 ). We trained a single RL policy using proximal policy optimization with collective thrust and body-rate control using a high-fidelity physics simulator based on the SimpleFlight framework. Real-world experimental results on the Crazyflie 2.1 platform show that the single RL policy successfully generalizes to different trajectory profiles and velocity references and maintains stability under a crosswind disturbance of up to 1.0 m s −1 which is a substantial challenge for this small class platform and even smaller payload underneath, where such aerodynamic forces are significant compared to the available control authority and system mass. Furthermore, the single RL policy was systematically evaluated using the mean Euclidean distance error, cable length transitions, and swing angle distributions. Although the single RL policy maintained a robust control performance, the experimental results indicated performance degradation at higher velocities owing to increased dynamic challenges such as nonlinear aerodynamic drag and actuator saturation. This study provides a detailed performance characterization that highlights the generalization capability of a single-payload-aware RL policy in real-world applications and the limitations arising from the hybrid dynamics of the system.
Related
- Reinforcement learning–coupled neural ode modelling reveals ferroptotic resolution failure as a shared dynamical axis linking periodontitis and coronary atherogenesis
- Reinforcement learning as a foundation for psychodynamics
- Multiproperty optimization of therapeutic antibodies through sequence editing via large language model and reinforcement learning
- Discordance between actual care and recommendations by a reinforcement learning model for sepsis
- How Past Experiences Shape New Beliefs: Generalisation and Adaptation in Human Reinforcement Learning
BibTeX
@article{karasahin2026characterization,
title = {{Characterization of a fixed reinforcement learning policy for aerial robot with suspended payload under variable flight conditions}},
author = {Ali Tahir Karasahin and Ziniu Wu and Basaran Bahadir Kocer},
journal = {Scientific Reports},
year = {2026},
month = oct,
doi = {10.1038/s41598-026-46111-x},
url = {https://www.nature.com/articles/s41598-026-46111-x},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Characterization of a fixed reinforcement learning policy for aerial robot with suspended payload under variable flight conditions
AU - Ali Tahir Karasahin
AU - Ziniu Wu
AU - Basaran Bahadir Kocer
PY - 2026
DA - 2026/10/01
JO - Scientific Reports
DO - 10.1038/s41598-026-46111-x
UR - https://www.nature.com/articles/s41598-026-46111-x
AB - Flights with suspended payloads are particularly challenging because of their coupled dynamics, which lead to instability and increased sensitivity to disturbances. Although reinforcement learning (RL) has successfully achieved controller performance, the generalization and robustness of a single policy remain significant areas of investigation. In this study, we characterized the performance and robustness of a single RL policy for an aerial robot with different trajectory profiles, including a smooth, feasible lemniscate curve and a sharp-turning, infeasible pentagram, under varying velocity references (0.5 m s ^−1 and 1.0 m s ^−1 ) and crosswind disturbances (1.0 m s ^−1 ). We trained a single RL policy using proximal policy optimization with collective thrust and body-rate control using a high-fidelity physics simulator based on the SimpleFlight framework. Real-world experimental results on the Crazyflie 2.1 platform show that the single RL policy successfully generalizes to different trajectory profiles and velocity references and maintains stability under a crosswind disturbance of up to 1.0 m s ^−1 which is a substantial challenge for this small class platform and even smaller payload underneath, where such aerodynamic forces are significant compared to the available control authority and system mass. Furthermore, the single RL policy was systematically evaluated using the mean Euclidean distance error, cable length transitions, and swing angle distributions. Although the single RL policy maintained a robust control performance, the experimental results indicated performance degradation at higher velocities owing to increased dynamic challenges such as nonlinear aerodynamic drag and actuator saturation. This study provides a detailed performance characterization that highlights the generalization capability of a single-payload-aware RL policy in real-world applications and the limitations arising from the hybrid dynamics of the system.
SN - 2045-2322
ER -