New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes
Abstract
Impulsive control, in the context of continuous-time Markov decision processes (CTMDPs), refers to actions that instantaneously update the state of the process, leading to theoretical complications. Previous work has overcome these complications, but at the expense of approachability and ease of implementation: those models are mathematically complex and incompatible with pre-existing solution methods. This paper presents an alternative formulation of impulsive control for CTMDPs that is easier to employ in the modelling of real-world problems and can be solved by pre-existing solution methods. This formulation is then extended to allow for multiple objectives and decomposability. A decomposition-based heuristic method is developed that leverages Dantzig-Wolfe decomposition to find approximate Pareto fronts for large multi-objective decomposable impulsive CTMDPs. The modelling framework and solution methodology are applied to a dynamic maintenance problem to demonstrate their computational efficiency, solution space coverage, and solution quality.
Related
- Kriging and Gaussian processes: interconnected frameworks for spatial modelling and prediction
- Structured SDEs for Realized GARCH: Continuous-Time Volatility Modelling and Component-Level Analysis
- Continuous logistic Gaussian random measure fields for spatial distributional modelling
- Influence of continuous predictor modelling methods on prediction stability in clinical prediction model development: an empirical comparison using real clinical data
- Criteria for discrete decomposability and admissibility in the branching problem
BibTeX
@misc{fairley2026modelling,
title = {{New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes}},
author = {Luke Fairley and Rob Shone and Peter Jacko and Jefferson Huang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.20126},
eprint = {2610.20126},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.20126},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - New Modelling and Solution Frameworks for Multiple Objectives, Decomposability and Impulsivity in Continuous-time Markov Decision Processes
AU - Luke Fairley
AU - Rob Shone
AU - Peter Jacko
AU - Jefferson Huang
PY - 2026
DA - 2026/10/14
PB - arXiv
DO - 10.48550/arxiv.2610.20126
UR - https://arxiv.org/abs/2610.20126
AB - Impulsive control, in the context of continuous-time Markov decision processes (CTMDPs), refers to actions that instantaneously update the state of the process, leading to theoretical complications. Previous work has overcome these complications, but at the expense of approachability and ease of implementation: those models are mathematically complex and incompatible with pre-existing solution methods. This paper presents an alternative formulation of impulsive control for CTMDPs that is easier to employ in the modelling of real-world problems and can be solved by pre-existing solution methods. This formulation is then extended to allow for multiple objectives and decomposability. A decomposition-based heuristic method is developed that leverages Dantzig-Wolfe decomposition to find approximate Pareto fronts for large multi-objective decomposable impulsive CTMDPs. The modelling framework and solution methodology are applied to a dynamic maintenance problem to demonstrate their computational efficiency, solution space coverage, and solution quality.
ER -