Dynamic Programming for A Linear-Gaussian Dual Control Problem
Abstract
We consider an infinite-horizon discounted dynamic programming (DP) problem for a countable-innovation belief-state model. The model is a surrogate for a linear-Gaussian dual control problem involving an unknown parameter vector and Gaussian observation noise. The surrogate preserves the first two moments of the Gaussian innovation and the resulting posterior second-moment structure. We allow unbounded belief and control spaces and nonnegative stage costs with quadratic growth in the posterior mean and covariance. Using a posterior-moment weight, we show that the Bellman operator is a contraction in a weighted sup-norm space. We characterize the optimal cost as the unique solution of the Bellman equation in this space, establish the existence of optimal stationary policies. We further prove weighted sup-norm convergence of value iteration, policy iteration, and optimistic policy iteration under vanishing approximation errors. The convergence results under vanishing approximation errors rely on general weighted sup-norm contraction properties and are applicable beyond the particular dual control problem considered here.
Related
- Dual Effect and Certainty Equivalence Revisited for a Linear-Gaussian Dual Control Problem
- Linear Programming Upper Bounds for the Coil-in-the-Box Problem
- Direct Neural Dynamic Programming Adaptive Control of Knee and Ankle Joints in a Simulated Prosthetic Leg Model
- No Spurious Local Minima in Full-Order Linear Quadratic Gaussian Control
- Optimal Control of Dynamic Gas Networks Using Gaussian Processes with Chance-Constraint Reformulation
BibTeX
@misc{wang2026dynamic,
title = {{Dynamic Programming for A Linear-Gaussian Dual Control Problem}},
author = {Ying Wang and Yuchao Li and Håkan Hjalmarsson},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.18630},
eprint = {2610.18630},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.18630},
}
RIS
TY - UNPB
TI - Dynamic Programming for A Linear-Gaussian Dual Control Problem
AU - Ying Wang
AU - Yuchao Li
AU - Håkan Hjalmarsson
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.18630
UR - https://arxiv.org/abs/2610.18630
AB - We consider an infinite-horizon discounted dynamic programming (DP) problem for a countable-innovation belief-state model. The model is a surrogate for a linear-Gaussian dual control problem involving an unknown parameter vector and Gaussian observation noise. The surrogate preserves the first two moments of the Gaussian innovation and the resulting posterior second-moment structure. We allow unbounded belief and control spaces and nonnegative stage costs with quadratic growth in the posterior mean and covariance. Using a posterior-moment weight, we show that the Bellman operator is a contraction in a weighted sup-norm space. We characterize the optimal cost as the unique solution of the Bellman equation in this space, establish the existence of optimal stationary policies. We further prove weighted sup-norm convergence of value iteration, policy iteration, and optimistic policy iteration under vanishing approximation errors. The convergence results under vanishing approximation errors rely on general weighted sup-norm contraction properties and are applicable beyond the particular dual control problem considered here.
ER -