Continuous Policy and Value Iteration for Stochastic Control Problems with Inhomogeneous Synchronous Coupling
Abstract
We propose a continuous policy-value iteration framework for infinite horizon stochastic control problems, in which the policy evolves through Langevin type dynamics driven by the gradient of the inhomogeneous Hamiltonian, while the value function is updated simultaneously with optimal policy iterations, avoiding directly solving its corresponding PDE at each iteration. The iteration scheme enjoys policy improvement under suitable regularity and admissibility assumptions. We show the exponential contraction of the joint policy value dynamics in Wasserstein-2 distance under the Lipschitz condition in expectation via the inhomogeneous synchronous coupling and verify these conditions for linear quadratic problems. Numerical experiments on linear quadratic models and the optimal consumption-investment problems in an incomplete market factor model illustrate policy improvement, convergence toward optimal policies, and the effect of decaying exploration noise.
Related
- Amortized Score-Hamiltonian Policy Iteration: A Grid-Free Scheme for Relaxed Stochastic Control Problems
- Solving Large-Scale Sparse Linear Complementarity Problems Using a Relaxed Shifted Matrix-Splitting Modulus-Based Iteration Method
- Periodic measures for time-inhomogeneous stochastic differential equations driven by multiplicative α-stable processes: Existence, uniqueness and numerical approximation
- Stochastic Inertial Krasnosel'skii-Mann Iteration Achieves Near-Optimal Sample Complexity
- Parameter-dependent inhomogeneous boundary-value problems in spaces of continuously differentiable functions
BibTeX
@misc{feng2026continuous,
title = {{Continuous Policy and Value Iteration for Stochastic Control Problems with Inhomogeneous Synchronous Coupling}},
author = {Qi Feng and Renjie Shao and Gu Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.10744},
eprint = {2610.10744},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10744},
}
RIS
TY - UNPB
TI - Continuous Policy and Value Iteration for Stochastic Control Problems with Inhomogeneous Synchronous Coupling
AU - Qi Feng
AU - Renjie Shao
AU - Gu Wang
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.10744
UR - https://arxiv.org/abs/2610.10744
AB - We propose a continuous policy-value iteration framework for infinite horizon stochastic control problems, in which the policy evolves through Langevin type dynamics driven by the gradient of the inhomogeneous Hamiltonian, while the value function is updated simultaneously with optimal policy iterations, avoiding directly solving its corresponding PDE at each iteration. The iteration scheme enjoys policy improvement under suitable regularity and admissibility assumptions. We show the exponential contraction of the joint policy value dynamics in Wasserstein-2 distance under the Lipschitz condition in expectation via the inhomogeneous synchronous coupling and verify these conditions for linear quadratic problems. Numerical experiments on linear quadratic models and the optimal consumption-investment problems in an incomplete market factor model illustrate policy improvement, convergence toward optimal policies, and the effect of decaying exploration noise.
ER -