Adaptive dynamic programming using Lyapunov function constraints
Abstract
This work is concerned with a stabilizing adaptive dynamic programming (ADP) approach to approximate solution of a given infinite-horizon optimal control problem. Since the latter problem cannot, in general, be solved exactly, a parametrized function approximator for the infinite-horizon cost function is introduced in ADP (so called ``critic''). This critic is used to adapt the parameters of the function approximator. The so called ``actor'' in turn derives the optimal input of the system. It is a notoriously hard problem to guarantee closed-loop stability of ADP due to the use of approximation structures in the control scheme. Since at least stabilizability is always assumed in the analyses of ADP, it is justified to invoke a respective Lyapunov function. The proposed ADP scheme explicitly uses the said Lyapunov function to simultaneously optimize the critic and guarantee closed-loop stability. A Hessian-free optimization routine is utilized for the actor and critic optimization problems. Convergence to prescribed vicinities of the optima is shown. A computational study showed significant performance improvement for the critic-based approach compared a nominal stabilizing controller for a range of initial conditions.
Versions
Related
- Direct Neural Dynamic Programming Adaptive Control of Knee and Ankle Joints in a Simulated Prosthetic Leg Model
- A sequential quadratic programming method for optimization with expectation objective functions, deterministic inequality constraints and robust subproblems
- Dynamic output-feedback design for generalized Lyapunov inequalities
- A Bregman proximal linearized ADMM for fractional programming with nonlinear coupling constraints
- Dynamic Quantum-Assisted Co-Design of Controller and Lyapunov Candidate Parameters for Nonlinear Systems
BibTeX
@misc{gohrt2026adaptive,
title = {{Adaptive dynamic programming using Lyapunov function constraints}},
author = {Thomas Göhrt and Pavel Osinenko and Stefan Streif},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.12110},
eprint = {2610.12110},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.12110},
}
RIS
TY - UNPB
TI - Adaptive dynamic programming using Lyapunov function constraints
AU - Thomas Göhrt
AU - Pavel Osinenko
AU - Stefan Streif
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2610.12110
UR - https://arxiv.org/abs/2610.12110
AB - This work is concerned with a stabilizing adaptive dynamic programming (ADP) approach to approximate solution of a given infinite-horizon optimal control problem. Since the latter problem cannot, in general, be solved exactly, a parametrized function approximator for the infinite-horizon cost function is introduced in ADP (so called ``critic''). This critic is used to adapt the parameters of the function approximator. The so called ``actor'' in turn derives the optimal input of the system. It is a notoriously hard problem to guarantee closed-loop stability of ADP due to the use of approximation structures in the control scheme. Since at least stabilizability is always assumed in the analyses of ADP, it is justified to invoke a respective Lyapunov function. The proposed ADP scheme explicitly uses the said Lyapunov function to simultaneously optimize the critic and guarantee closed-loop stability. A Hessian-free optimization routine is utilized for the actor and critic optimization problems. Convergence to prescribed vicinities of the optima is shown. A computational study showed significant performance improvement for the critic-based approach compared a nominal stabilizing controller for a range of initial conditions.
ER -