Reinforcement learning for constrained stochastic linear--quadratic control in continuous time
Abstract
We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.
Related
- Continuous Delayed-Memory Stochastic Gradient Descent and Continuous-Time Reinforcement Learning from History of Astrophysical Time Series Studies
- Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies
- Model-free Reinforcement Learning for Continuous Time and State: A Stochastic Maximum Principle Approach
- Adaptive Variance-Reduced Stochastic Sequential (Quadratic Constrained) Quadratic Programming
- Continuous-Time Constrained Linear Quadratic Regulator for Switched Linear Systems
BibTeX
@misc{wu2026reinforcement,
title = {{Reinforcement learning for constrained stochastic linear--quadratic control in continuous time}},
author = {Weiping Wu and Qitong Pan and Xun Li and Peng Wu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34995},
eprint = {2610.34995},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34995},
}
RIS
TY - UNPB
TI - Reinforcement learning for constrained stochastic linear--quadratic control in continuous time
AU - Weiping Wu
AU - Qitong Pan
AU - Xun Li
AU - Peng Wu
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.34995
UR - https://arxiv.org/abs/2610.34995
AB - We study continuous-time stochastic linear--quadratic control with a scalar state and nonnegative control constraints. With quadratic running costs, randomized exploration can change the optimal mean feedback. We introduce an auxiliary problem without entropy in which the conditional policy mean is nonnegative and the covariance is prescribed in proportion to the squared state. Moment reduction and state separation yield two regional Riccati equations and characterize the finite-horizon value function and optimal conditional mean. We give a sufficient control--cost compatibility condition under which this mean coincides with the optimal feedback of the original constrained problem. For time-invariant coefficients, we characterize the undiscounted infinite-horizon optimum and establish mean square asymptotic stability under the stated assumptions. For unknown dynamics, we develop model-free actor--critic schemes for finite and infinite horizons. Numerical experiments compare the learned policies with their analytical targets and examine the effect of exploration on the optimal mean.
ER -