Pseudo Closed-Loop Bootstrapping for Reinforcement Learning Based AO Control
Abstract
High contrast imaging with ground-based telescopes requires extremely precise wavefront control. The performance of the controller ultimately depends on the quality and efficiency of the wavefront sensing technique, which has led to the development of ever more sensitive wavefront sensors (WFS). However, the increase in sensitivity comes at the cost of greater nonlinearity, which poses challenges for conventional linear reconstructors and controllers. Reinforcement Learning (RL) is a branch of machine learning in which control policies are learned by interacting with the environment. RL has recently attracted interest in the field of XAO, and previous studies have demonstrated that a model-based RL variant, the Policy Optimization for Adaptive Optics (PO4AO), can effectively compensate for temporal delays, misregistration errors, and moderate WFS nonlinearities. PO4AO learns an initial control strategy by collecting closed-loop data from the integrator controller with a linear reconstructor. However, in cases of highly non-linear WFS and challenging conditions, closing the loop with the integrator is difficult, and high-fidelity data cannot be collected. Consequently, learning robust control with PO4AO is challenging. We propose a robust learning strategy in which PO4AO is pre-calibrated using an internal light source and a DM, enabling direct on-sky closed-loop operation without a stable integrator controller for bootstrapping the initial models.
Related
- Exploring reinforcement learning to enhance focal-plane wavefront control for vortex coronagraphs
- Reinforcement learning for post-coronagraphic wavefront control
- aRieL - Reinforcement Learning for the Ariel Space Telescope Scheduling
- Transient PULSE: Code for the Transient Propagation of UltraLight (pseudo)Scalars to Earth
- Hyperbolic-Type Solitons on Ricci Pseudo-Symmetric Riemannian Concircular Structure Manifolds
BibTeX
@misc{halonen2026pseudo,
title = {{Pseudo Closed-Loop Bootstrapping for Reinforcement Learning Based AO Control}},
author = {Eemil Halonen and Jalo Nousiainen and Markus Kasper and Nuutti Hyvönen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.11463},
eprint = {2610.11463},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.11463},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Pseudo Closed-Loop Bootstrapping for Reinforcement Learning Based AO Control
AU - Eemil Halonen
AU - Jalo Nousiainen
AU - Markus Kasper
AU - Nuutti Hyvönen
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2610.11463
UR - https://arxiv.org/abs/2610.11463
AB - High contrast imaging with ground-based telescopes requires extremely precise wavefront control. The performance of the controller ultimately depends on the quality and efficiency of the wavefront sensing technique, which has led to the development of ever more sensitive wavefront sensors (WFS). However, the increase in sensitivity comes at the cost of greater nonlinearity, which poses challenges for conventional linear reconstructors and controllers. Reinforcement Learning (RL) is a branch of machine learning in which control policies are learned by interacting with the environment. RL has recently attracted interest in the field of XAO, and previous studies have demonstrated that a model-based RL variant, the Policy Optimization for Adaptive Optics (PO4AO), can effectively compensate for temporal delays, misregistration errors, and moderate WFS nonlinearities. PO4AO learns an initial control strategy by collecting closed-loop data from the integrator controller with a linear reconstructor. However, in cases of highly non-linear WFS and challenging conditions, closing the loop with the integrator is difficult, and high-fidelity data cannot be collected. Consequently, learning robust control with PO4AO is challenging. We propose a robust learning strategy in which PO4AO is pre-calibrated using an internal light source and a DM, enabling direct on-sky closed-loop operation without a stable integrator controller for bootstrapping the initial models.
ER -