Offline Approximate Value Iteration: A Reinforcement Learning Approach for Satellite Constellation Deployment
Abstract
This paper introduces a reinforcement learning-based approximate dynamic programming approach to optimize the deployment of Walker-type satellite constellations under uncertainties, including launch failures and satellite malfunctions. We formulate the problem as a Markov Decision Process that incorporates low-injection orbit launches, continuous low-thrust orbit raising, and variable satellite loads. We adopt the offline approximate value iteration solution method, using linear function approximation with basis functions and action pruning strategies to substantially reduce the computational and memory requirements for finding optimal decision policies. Preliminary results demonstrate convergence to policies with low expected costs and deployment times, both of which are lower than those of a rolling-horizon baseline, showcasing a novel application of approximate dynamic programming in space logistics.
Related
- A two-stage approach to satellite constellation optimization: classical and QUBO formulations
- LEO Constellation Analysis for Satellite IoT: An Analysis of Coverage Gaps and Retransmissions
- On the Lack of Periodicity of Walker Satellite Constellation Routing Tables
- Causal DAG-Guided Reinforcement Learning for Long-Term Physical Activity Maintenance: Using Prior Data to Inform Future Deployment
- Fluid Antenna Secure Channel-driven Constellation (FASCC): a Physical Layer Security approach
BibTeX
@misc{leuridan2026offline,
title = {{Offline Approximate Value Iteration: A Reinforcement Learning Approach for Satellite Constellation Deployment}},
author = {Tristan Leuridan and Yiping Fang and Adam Abdin},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.02809},
eprint = {2611.02809},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.02809},
}
RIS
TY - UNPB
TI - Offline Approximate Value Iteration: A Reinforcement Learning Approach for Satellite Constellation Deployment
AU - Tristan Leuridan
AU - Yiping Fang
AU - Adam Abdin
PY - 2026
DA - 2026/11/02
PB - arXiv
DO - 10.48550/arxiv.2611.02809
UR - https://arxiv.org/abs/2611.02809
AB - This paper introduces a reinforcement learning-based approximate dynamic programming approach to optimize the deployment of Walker-type satellite constellations under uncertainties, including launch failures and satellite malfunctions. We formulate the problem as a Markov Decision Process that incorporates low-injection orbit launches, continuous low-thrust orbit raising, and variable satellite loads. We adopt the offline approximate value iteration solution method, using linear function approximation with basis functions and action pruning strategies to substantially reduce the computational and memory requirements for finding optimal decision policies. Preliminary results demonstrate convergence to policies with low expected costs and deployment times, both of which are lower than those of a rolling-horizon baseline, showcasing a novel application of approximate dynamic programming in space logistics.
ER -