Soft-Update MSA for Nonconvex Stochastic Optimal Control: Theory and Algorithms
Abstract
For the method of successive approximations (MSA), a fundamental challenge is that the stochastic maximum principle (SMP) provides an optimality condition rather than a descent rule. To overcome this difficulty for nonconvex stochastic control problems, we develop a soft-update MSA (SU-MSA) with a second-order-adjoint correction of the Hamiltonian and a neural realization called DeepSMP. Both an explicit stepsize rule guaranteeing cost descent and the convergence of the algorithm are established. Moreover, the global SMP condition is asymptotically satisfied when the corrected Hamiltonian is convex, while in a special case the cost is further proved to converge to the global optimum. To extend SU-MSA to high dimensions, we develop DeepSMP using a differentiable smooth approximation of the corrected Hamiltonian and analyze the convergence of the resulting objective and its gradient. Numerical experiments demonstrate the effectiveness of the proposed framework.
Related
- Optimal Algorithms for Nonsmooth Nonconvex Stochastic Optimization with Constraints
- Douglas-Rachford splitting method for nonconvex stochastic optimal control
- Lower Bounds for Stochastic First-Order Algorithms with Variance Reduction in Nonconvex--Concave Minimax Optimization
- Necessary and sufficient conditions for optimal control of coupled forward-backward stochastic control systems driven by G-Brownian motion
- Quantum algorithms for optimal control: large-scale finite-horizon linear-quadratic regulation
BibTeX
@misc{ji2026softupdate,
title = {{Soft-Update MSA for Nonconvex Stochastic Optimal Control: Theory and Algorithms}},
author = {Shaolin Ji and Xinmeng Liu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.30585},
eprint = {2610.30585},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.30585},
}
RIS
TY - UNPB
TI - Soft-Update MSA for Nonconvex Stochastic Optimal Control: Theory and Algorithms
AU - Shaolin Ji
AU - Xinmeng Liu
PY - 2026
DA - 2026/10/22
PB - arXiv
DO - 10.48550/arxiv.2610.30585
UR - https://arxiv.org/abs/2610.30585
AB - For the method of successive approximations (MSA), a fundamental challenge is that the stochastic maximum principle (SMP) provides an optimality condition rather than a descent rule. To overcome this difficulty for nonconvex stochastic control problems, we develop a soft-update MSA (SU-MSA) with a second-order-adjoint correction of the Hamiltonian and a neural realization called DeepSMP. Both an explicit stepsize rule guaranteeing cost descent and the convergence of the algorithm are established. Moreover, the global SMP condition is asymptotically satisfied when the corrected Hamiltonian is convex, while in a special case the cost is further proved to converge to the global optimum. To extend SU-MSA to high dimensions, we develop DeepSMP using a differentiable smooth approximation of the corrected Hamiltonian and analyze the convergence of the resulting objective and its gradient. Numerical experiments demonstrate the effectiveness of the proposed framework.
ER -