Convex geometry of diffusion laws and gradient flows for stochastic control
Abstract
We study gradient-type dynamics for weak stochastic control through the laws of controlled diffusions on path space. Although the objective is generally nonconvex in the drift, a uniformly convex running cost induces a strictly convex functional of the corresponding path law and hence a natural Bregman geometry. We construct the associated implicit proximal scheme and prove its well-posedness and convergence to a continuous-time mirror flow. When the mirror cost coincides with the running cost and the law-dependent potential is linear, the flow becomes affine in centered entropy coordinates. This yields a variational construction of the resulting Newton flow, uniform stability estimates, and exponential convergence to the optimizer. For a general uniformly convex mirror cost and a nonlinear convex potential of the law, we construct the natural-gradient flow globally in learning time using BMO estimates for quadratic BSDEs, and establish quantitative convergence and a first-order discretization error. We further analyze interacting finite-particle approximations, proving exponential convergence to the finite-particle optimum and an O(1/N) normalized discrepancy from the mean-field optimum. In the Markovian setting, reflection coupling yields uniform spatial regularity and exponential convergence of the feedback controls on Rd, including for nonquadratic state-dependent Hamiltonians.
Related
- Gradient-Free Methods for Stochastic Convex Optimization with Stochastic Functional Constraints
- Exact Theory and Optimal Stepsizes for Stochastic Gradient Descent in Smooth Convex Minimization
- The Exact Time-Uniform Rate Frontier for Stochastic Gradient Descent on Smooth Convex Objectives
- Adaptive Restart under Unknown KL Geometry in Stochastic Weakly-Convex Optimization
- Random Reshuffling for Smooth Convex Optimization: Dominates Stochastic Gradient Descent
BibTeX
@misc{bai2026convex,
title = {{Convex geometry of diffusion laws and gradient flows for stochastic control}},
author = {Yupeng Bai and Louis-Pierre Chaintron and Zhenjie Ren and Songbo Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.03487},
eprint = {2610.03487},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.03487},
}
RIS
TY - UNPB
TI - Convex geometry of diffusion laws and gradient flows for stochastic control
AU - Yupeng Bai
AU - Louis-Pierre Chaintron
AU - Zhenjie Ren
AU - Songbo Wang
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2610.03487
UR - https://arxiv.org/abs/2610.03487
AB - We study gradient-type dynamics for weak stochastic control through the laws of controlled diffusions on path space. Although the objective is generally nonconvex in the drift, a uniformly convex running cost induces a strictly convex functional of the corresponding path law and hence a natural Bregman geometry. We construct the associated implicit proximal scheme and prove its well-posedness and convergence to a continuous-time mirror flow. When the mirror cost coincides with the running cost and the law-dependent potential is linear, the flow becomes affine in centered entropy coordinates. This yields a variational construction of the resulting Newton flow, uniform stability estimates, and exponential convergence to the optimizer. For a general uniformly convex mirror cost and a nonlinear convex potential of the law, we construct the natural-gradient flow globally in learning time using BMO estimates for quadratic BSDEs, and establish quantitative convergence and a first-order discretization error. We further analyze interacting finite-particle approximations, proving exponential convergence to the finite-particle optimum and an O(1/N) normalized discrepancy from the mean-field optimum. In the Markovian setting, reflection coupling yields uniform spatial regularity and exponential convergence of the feedback controls on Rd, including for nonquadratic state-dependent Hamiltonians.
ER -