Reinforcement-learning control of turbulence transition in the modified Hasegawa-Wakatani system
Abstract
Control of plasma turbulence remains a long-standing challenge in magnetically confined fusion research. Here, we explore a reinforcement-learning (RL) approach for bidirectional control of the turbulence-zonal-flow transition in the modified Hasegawa-Wakatani system, a minimal model of electrostatic drift-wave turbulence. Within our model, a weak zonal drag damps the otherwise long-lived zonal structures, giving them a finite lifetime and restoring the drive-damping balance required for repeatable transitions within finite control episodes; meanwhile, actuation is applied through a spatially distributed Gaussian source field under a time-weighted budget constraint on actuation costs. The plasma model is then coupled to CNN-based soft actor-critic and twin-delayed deterministic policy-gradient agents through a GPU-native JAX solver that is highly optimized for fast online training. In the turbulence-suppression task, the learned budget-aware schedule achieves the lowest time-integrated turbulent flux for a given consumed budget, outperforming both constant and linearly decreasing baselines across all tested unseen initial conditions. In the inverse zonal-break task, the agent discovers an up-down antisymmetric actuation pattern that induces radial E× B convection, disrupts the zonal structure, and sustains the turbulent state. A physics-informed warm-buffer initialization facilitates this discovery, as random exploration alone struggles to locate the narrow optimal manifold within the vast action space. These results demonstrate reinforcement learning as a practical trajectory optimizer for nonlinear plasma dynamics, such as turbulence control, and highlight the importance of physics guidance in such applications.
Related
- Integrable turbulence and statistical properties of chaotic wave fields in the complex modified Korteweg-de Vries equation
- On Phase Transition of ITG Turbulence in the Dimits shift
- A high-fidelity numerical database for free-stream-turbulence-induced transition under varying pressure gradients
- Transition from a weak-elastic to an elastic-activated regime in two-dimensional polymeric turbulence
- Fully developed active turbulence defined through a non-equilibrium phase transition
BibTeX
@misc{sun2026reinforcementlearning,
title = {{Reinforcement-learning control of turbulence transition in the modified Hasegawa-Wakatani system}},
author = {Luning Sun and Ben Zhu and Xin-Yang Liu and Deepak Akhare and Jian-Xun Wang},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.27845},
eprint = {2608.27845},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.27845},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Reinforcement-learning control of turbulence transition in the modified Hasegawa-Wakatani system
AU - Luning Sun
AU - Ben Zhu
AU - Xin-Yang Liu
AU - Deepak Akhare
AU - Jian-Xun Wang
PY - 2026
DA - 2026/08/28
PB - arXiv
DO - 10.48550/arxiv.2608.27845
UR - https://arxiv.org/abs/2608.27845
AB - Control of plasma turbulence remains a long-standing challenge in magnetically confined fusion research. Here, we explore a reinforcement-learning (RL) approach for bidirectional control of the turbulence-zonal-flow transition in the modified Hasegawa-Wakatani system, a minimal model of electrostatic drift-wave turbulence. Within our model, a weak zonal drag damps the otherwise long-lived zonal structures, giving them a finite lifetime and restoring the drive-damping balance required for repeatable transitions within finite control episodes; meanwhile, actuation is applied through a spatially distributed Gaussian source field under a time-weighted budget constraint on actuation costs. The plasma model is then coupled to CNN-based soft actor-critic and twin-delayed deterministic policy-gradient agents through a GPU-native JAX solver that is highly optimized for fast online training. In the turbulence-suppression task, the learned budget-aware schedule achieves the lowest time-integrated turbulent flux for a given consumed budget, outperforming both constant and linearly decreasing baselines across all tested unseen initial conditions. In the inverse zonal-break task, the agent discovers an up-down antisymmetric actuation pattern that induces radial E× B convection, disrupts the zonal structure, and sustains the turbulent state. A physics-informed warm-buffer initialization facilitates this discovery, as random exploration alone struggles to locate the narrow optimal manifold within the vast action space. These results demonstrate reinforcement learning as a practical trajectory optimizer for nonlinear plasma dynamics, such as turbulence control, and highlight the importance of physics guidance in such applications.
ER -