Global Sobolev Convergence of Howard Iteration for Finite-Horizon Controlled Diffusions
Abstract
We study Howard policy iteration for finite-horizon stochastic control problems in which the control enters the diffusion coefficient. Each iteration selects a policy by maximizing the Hamiltonian and evaluates it through a linear parabolic equation. Since the improvement depends on the Hessian, even smooth data can produce discontinuous evaluation coefficients. We prove global convergence in the full parabolic Sobolev space under assumptions that allow this loss of regularity. In one space dimension, bounded measurable coefficients and uniform ellipticity suffice, without a large discount, a short horizon, small diffusion variation, or regularity assumptions on the improving policies. Multidimensional extensions hold under structural conditions on the diffusion matrix. The proof combines monotonicity with a parabolic growth estimate to show that the Bellman residual vanishes in measure. Uniform higher integrability then yields strong convergence of the values, gradients, Hessians, and time derivatives, and constructs the unique strong solution. We also establish residual error bounds, extend the argument to entropy regularization and zero-temperature limits, and obtain quadratic convergence for special one-dimensional models.
Related
- Exponential rate of convergence of relative value iteration algorithms for ergodic controls of diffusions
- Uniqueness of Extremizers and Convergence of the Euler-Lagrange Iteration for the Critical Hardy-Littlewood-Sobolev Inequality
- Quantitative Convergence of Markov Equilibria for Time-Inconsistent Controlled Diffusions
- SDDP has polynomial iteration complexity in the horizon
- Convergence and acceleration of a nonlinear fixed-point iteration for computing the Fitness Centrality of general graphs
BibTeX
@misc{miao2026global,
title = {{Global Sobolev Convergence of Howard Iteration for Finite-Horizon Controlled Diffusions}},
author = {Linyu Miao},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.15607},
eprint = {2609.15607},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.15607},
}
RIS
TY - UNPB
TI - Global Sobolev Convergence of Howard Iteration for Finite-Horizon Controlled Diffusions
AU - Linyu Miao
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2609.15607
UR - https://arxiv.org/abs/2609.15607
AB - We study Howard policy iteration for finite-horizon stochastic control problems in which the control enters the diffusion coefficient. Each iteration selects a policy by maximizing the Hamiltonian and evaluates it through a linear parabolic equation. Since the improvement depends on the Hessian, even smooth data can produce discontinuous evaluation coefficients. We prove global convergence in the full parabolic Sobolev space under assumptions that allow this loss of regularity. In one space dimension, bounded measurable coefficients and uniform ellipticity suffice, without a large discount, a short horizon, small diffusion variation, or regularity assumptions on the improving policies. Multidimensional extensions hold under structural conditions on the diffusion matrix. The proof combines monotonicity with a parabolic growth estimate to show that the Bellman residual vanishes in measure. Uniform higher integrability then yields strong convergence of the values, gradients, Hessians, and time derivatives, and constructs the unique strong solution. We also establish residual error bounds, extend the argument to entropy regularization and zero-temperature limits, and obtain quadratic convergence for special one-dimensional models.
ER -