Reinforcement Learning Controllers for Fusion Scenario Development
Abstract
Scenario development, the design of the waveforms that take a tokamak discharge from breakdown to flat-top, is still an iterative, labour-intensive process. Every candidate scenario has to be simulated with a controller in the loop, and the classical shape and current controllers need low-level actuator references. So the process is iterative and includes a number of trial-and-error steps. We propose to replace the per-scenario controller tuning by a single goal-conditioned reinforcement-learning controller trained once, in advance, against a free-boundary equilibrium and transport simulator. A scenario is then specified only by the target waveforms of the physical quantities of interest (plasma current, line-averaged electron density, LCFS shape, x-point positions), and the controller produces the coil-current and fueling commands that realize them. We demonstrate the approach on an EAST plasma-current ramp-up: the controller takes over a small limited plasma, ramps the current and the density, grows the cross-section, forms two x-points and holds a double-null diverted configuration against sampled shape targets for the rest of a 1.2 s episode. The neural network controller is trained using TQC algorithm. On 1024 test scenarios sampled from a low-discrepancy scrambled Sobol sequence in 11-dimensional target space, the controller forms the divertor in 86% of the cases, holds it for more than 0.5 s in 67% and reaches the end of the episode in a diverted state in 58%, with flat-top tracking errors of a few kA in plasma current, ∼ 0.15×1019 m−3 in density, ∼ 2 cm in position and ∼ 8 cm in x-point location; the failures concentrate at the high-current density (J) region which is believed to be physically or from engineering perspective not feasible within ohmic mode.
Related
- Physics basis for the reference flat-top plasma scenario in the ST–E1 fusion power plant
- Fabric retrofit scenario analysis under uncertainty for an urban housing stock using machine learning
- Reinforcement learning for robust calibration of multiqudit quantum gates
- Discovering network resilience formulas with symbolized reinforcement learning
- Reinforcement learning to disentangle multiqubit quantum states from partial observations
BibTeX
@misc{adishchev2026reinforcement,
title = {{Reinforcement Learning Controllers for Fusion Scenario Development}},
author = {Evgeny Adishchev and Maxim Nurgaliev and Dmitry Sorokin and Alexander Granovsky and Maksim Stokolesov and Igor Prokofyev and Eduard Khairutdinov and Georgy Subbotin},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.26138},
eprint = {2610.26138},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.26138},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Reinforcement Learning Controllers for Fusion Scenario Development
AU - Evgeny Adishchev
AU - Maxim Nurgaliev
AU - Dmitry Sorokin
AU - Alexander Granovsky
AU - Maksim Stokolesov
AU - Igor Prokofyev
AU - Eduard Khairutdinov
AU - Georgy Subbotin
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.26138
UR - https://arxiv.org/abs/2610.26138
AB - Scenario development, the design of the waveforms that take a tokamak discharge from breakdown to flat-top, is still an iterative, labour-intensive process. Every candidate scenario has to be simulated with a controller in the loop, and the classical shape and current controllers need low-level actuator references. So the process is iterative and includes a number of trial-and-error steps. We propose to replace the per-scenario controller tuning by a single goal-conditioned reinforcement-learning controller trained once, in advance, against a free-boundary equilibrium and transport simulator. A scenario is then specified only by the target waveforms of the physical quantities of interest (plasma current, line-averaged electron density, LCFS shape, x-point positions), and the controller produces the coil-current and fueling commands that realize them. We demonstrate the approach on an EAST plasma-current ramp-up: the controller takes over a small limited plasma, ramps the current and the density, grows the cross-section, forms two x-points and holds a double-null diverted configuration against sampled shape targets for the rest of a 1.2 s episode. The neural network controller is trained using TQC algorithm. On 1024 test scenarios sampled from a low-discrepancy scrambled Sobol sequence in 11-dimensional target space, the controller forms the divertor in 86% of the cases, holds it for more than 0.5 s in 67% and reaches the end of the episode in a diverted state in 58%, with flat-top tracking errors of a few kA in plasma current, ∼ 0.15×10^19 m^−3 in density, ∼ 2 cm in position and ∼ 8 cm in x-point location; the failures concentrate at the high-current density (J) region which is believed to be physically or from engineering perspective not feasible within ohmic mode.
ER -