Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations
Abstract
We analyze a neural semi-discrete method for high-dimensional first-order Hamilton-Jacobi-Bellman (HJB) equations with known or learned dynamics. Centered differences and an artificial viscosity Nh=O(h) define a monotone operator evaluated through 2d+1 shifted network queries; policy iteration solves the resulting Bellman equation without a tensor grid. At fixed h, the sharp componentwise condition maxi|fi|≤2N turns every frozen-policy operator into a nearest-neighbor Markov-chain generator with a policy-independent total jump rate. Uniformization gives whole-space well-posedness for measurable feedbacks, an explicit Poisson-tail bound on the numerical domain of dependence, and boundary-free localization. The representation also yields a posteriori policy-evaluation bounds that account for residual and learned-model errors. A greedy-gap analysis controls inexact policy iteration at fixed h; a separate consistency estimate connects the semi-discrete equation to the continuous HJB equation. Experiments reproduce the extremal tail, show rates consistent with O(√h) and nearly h-independent exact-policy-iteration decay, and assess empirical estimator effectivity. A nonsmooth example shows that the continuous residual can miss a non-viscosity solution, whereas the shifted residual detects the defect. Further tests provide a structured interval-verified certificate calibration, an early-budget benefit of policy freezing for bang-bang control, and learned-dynamics diagnostics. A structured nonlinear problem with active compact-control constraints is tested against a manufactured semi-discrete reference through d=1024.
Related
- Policy iteration for Hamilton-Jacobi-Isaacs equations with control constraints and comparison with Hamilton-Jacobi-Bellman equations
- The Effect of Quadrature on the Convergence of Policy Iteration for Hamilton-Jacobi-Bellman Equations
- Analytical and numerical periodic homogenization of Hamilton--Jacobi--Bellman and Isaacs equations in a W2,p Cordes framework
- Variable-Coefficient Parabolic Equations and Finite-Horizon Hamilton--Jacobi--Bellman Equations in Augmented Spectral Barron Spaces
- Error Bounds for Boundary-Stopped Positive Cubature Schemes for Hamilton--Jacobi--Bellman Equations
BibTeX
@misc{kim2026monotone,
title = {{Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations}},
author = {Minseok Kim and Yeongjong Kim and Namkyeong Cho and Yeoneung Kim},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.07116},
eprint = {2605.07116},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.07116},
}
RIS
TY - UNPB
TI - Monotone Neural Policy Iteration for High-Dimensional First-Order Hamilton--Jacobi--Bellman Equations
AU - Minseok Kim
AU - Yeongjong Kim
AU - Namkyeong Cho
AU - Yeoneung Kim
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2605.07116
UR - https://arxiv.org/abs/2605.07116
AB - We analyze a neural semi-discrete method for high-dimensional first-order Hamilton-Jacobi-Bellman (HJB) equations with known or learned dynamics. Centered differences and an artificial viscosity Nh=O(h) define a monotone operator evaluated through 2d+1 shifted network queries; policy iteration solves the resulting Bellman equation without a tensor grid. At fixed h, the sharp componentwise condition max_i|f_i|≤2N turns every frozen-policy operator into a nearest-neighbor Markov-chain generator with a policy-independent total jump rate. Uniformization gives whole-space well-posedness for measurable feedbacks, an explicit Poisson-tail bound on the numerical domain of dependence, and boundary-free localization. The representation also yields a posteriori policy-evaluation bounds that account for residual and learned-model errors. A greedy-gap analysis controls inexact policy iteration at fixed h; a separate consistency estimate connects the semi-discrete equation to the continuous HJB equation. Experiments reproduce the extremal tail, show rates consistent with O(√h) and nearly h-independent exact-policy-iteration decay, and assess empirical estimator effectivity. A nonsmooth example shows that the continuous residual can miss a non-viscosity solution, whereas the shifted residual detects the defect. Further tests provide a structured interval-verified certificate calibration, an early-budget benefit of policy freezing for bang-bang control, and learned-dynamics diagnostics. A structured nonlinear problem with active compact-control constraints is tested against a manufactured semi-discrete reference through d=1024.
ER -