Dynamic Programming for Value-at-Risk Markov Decision Processes in Continuous Spaces
Abstract
Value-at-Risk (VaR) is a widely adopted quantile-based risk measure in practice, yet incorporating VaR objectives into Markov decision processes (VaR MDPs) remains a longstanding challenge. Recent work has derived dynamic programming equations for VaR MDPs in finite state and action spaces but left the case of more general spaces unresolved. This paper establishes, for the first time, the validity of these equations in continuous spaces, for both finite- and infinite-horizon settings. Such cases naturally arise in many real-world dynamic decision-making problems. As an application, we study an optimal execution problem with a VaR objective and demonstrate that our framework enables the characterization of optimal dynamic strategies, resolving an open question in the field.
Related
- Linear Programming Representations and Strongly Polynomial Algorithms for Robust Markov Decision Processes
- New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes
- Worst-Case Bayesian CVaR Regret Bounds for Episodic Markov Decision Processes
- Percolation Markov Decision Processes
- Policy Iteration Is Not Strongly Polynomial for Deterministic Markov Decision Processes: The Price of Algorithmic Anarchy
BibTeX
@misc{chen2026dynamic,
title = {{Dynamic Programming for Value-at-Risk Markov Decision Processes in Continuous Spaces}},
author = {Xian Chen and Erick Delage and Weikai Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.24439},
eprint = {2610.24439},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24439},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Dynamic Programming for Value-at-Risk Markov Decision Processes in Continuous Spaces
AU - Xian Chen
AU - Erick Delage
AU - Weikai Wang
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2610.24439
UR - https://arxiv.org/abs/2610.24439
AB - Value-at-Risk (VaR) is a widely adopted quantile-based risk measure in practice, yet incorporating VaR objectives into Markov decision processes (VaR MDPs) remains a longstanding challenge. Recent work has derived dynamic programming equations for VaR MDPs in finite state and action spaces but left the case of more general spaces unresolved. This paper establishes, for the first time, the validity of these equations in continuous spaces, for both finite- and infinite-horizon settings. Such cases naturally arise in many real-world dynamic decision-making problems. As an application, we study an optimal execution problem with a VaR objective and demonstrate that our framework enables the characterization of optimal dynamic strategies, resolving an open question in the field.
ER -