Turnpike Properties for Exploratory Stochastic Linear-Quadratic Optimal Control with Indefinite Cost Functionals
Abstract
Continuous-time exploratory stochastic control provides a stochastic control framework for the exploration--exploitation trade-off in reinforcement learning. In this paper, we study the long-time behavior of finite-horizon exploratory stochastic linear-quadratic (LQ) optimal control problems as the time horizon T→∞, and establish so-called turnpike properties for such problems. In the finite-horizon setting, we derive a feedback representation of the exploratory stochastic LQ problem and analyze its long-time structure. We formulate an exploratory static distribution optimization problem analogous to the classical static optimization problem and identify the corresponding static reference object. We prove a turnpike property for the optimal state process relative to a stochastic reference process, as well as a Wasserstein turnpike property for the optimal control distribution. In addition to the exploratory control formulation, another key feature of our results is that, by introducing a joint coercivity condition, we allow the cost functional to be indefinite and the controlled state system to be merely stabilizable. Consequently, the derived turnpike property is novel even within the classical stochastic LQ control framework.
Related
- Stochastic Riccati equations with jumps for indefinite linear-quadratic control: A stochastic value-flow approach
- General Leader-Follower Linear-Quadratic Stochastic Graphon Games with Indefinite Control Weights
- Recursive Stochastic Linear-Quadratic Control with Indefinite Weights for Jump-Diffusion Systems with Random Coefficients
- On the average of class numbers of indefinite binary quadratic forms
- On the Bach tensor and quadratic curvature functionals
BibTeX
@misc{hu2026turnpike,
title = {{Turnpike Properties for Exploratory Stochastic Linear-Quadratic Optimal Control with Indefinite Cost Functionals}},
author = {Yaohua Hu and Zhaoqing Li and Hanxiao Wang and Jie Xiong},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.41139},
eprint = {2610.41139},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.41139},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Turnpike Properties for Exploratory Stochastic Linear-Quadratic Optimal Control with Indefinite Cost Functionals
AU - Yaohua Hu
AU - Zhaoqing Li
AU - Hanxiao Wang
AU - Jie Xiong
PY - 2026
DA - 2026/10/30
PB - arXiv
DO - 10.48550/arxiv.2610.41139
UR - https://arxiv.org/abs/2610.41139
AB - Continuous-time exploratory stochastic control provides a stochastic control framework for the exploration--exploitation trade-off in reinforcement learning. In this paper, we study the long-time behavior of finite-horizon exploratory stochastic linear-quadratic (LQ) optimal control problems as the time horizon T→∞, and establish so-called turnpike properties for such problems. In the finite-horizon setting, we derive a feedback representation of the exploratory stochastic LQ problem and analyze its long-time structure. We formulate an exploratory static distribution optimization problem analogous to the classical static optimization problem and identify the corresponding static reference object. We prove a turnpike property for the optimal state process relative to a stochastic reference process, as well as a Wasserstein turnpike property for the optimal control distribution. In addition to the exploratory control formulation, another key feature of our results is that, by introducing a joint coercivity condition, we allow the cost functional to be indefinite and the controlled state system to be merely stabilizable. Consequently, the derived turnpike property is novel even within the classical stochastic LQ control framework.
ER -