Percolation Markov Decision Processes
Abstract
We study Percolation Markov Decision Processes (PMDPs), in which the decision maker repeatedly moves a token through the d-dimensional integer lattice with deterministic transitions and random payoffs assigned to the edges. Payoffs are revealed before the beginning of the decision process and the decision maker aims to maximize the average accumulated payoff over a fixed horizon. We establish the existence of the uniform value (as the horizon tends to infinity) and 0-optimal strategies. In the particular case of Bernoulli payoffs, we establish continuity results for the uniform value. We also present several one-dimensional examples to illustrate that optimal strategies may be very complex. Then, we introduce a dimensional lifting property that allows PMDPs to be approximated by PMDPs of lower dimension and could be used to approximate critical probability thresholds in percolation theory.
Related
- Worst-Case Bayesian CVaR Regret Bounds for Episodic Markov Decision Processes
- Dynamic Programming for Value-at-Risk Markov Decision Processes in Continuous Spaces
- Policy Iteration Is Not Strongly Polynomial for Deterministic Markov Decision Processes: The Price of Algorithmic Anarchy
- A Geometric Theory of Decision Boundaries in Structured Markov Decision Processes
- Differential Properties of Objective and Value Functions for Discounted Markov Decision Processes with Finite State and Action Sets
BibTeX
@misc{garcia2026percolation,
title = {{Percolation Markov Decision Processes}},
author = {Melissa González García and Guillaume Vigeral},
howpublished = {HAL},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.19905},
eprint = {2609.19905},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05752370v1},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Percolation Markov Decision Processes
AU - Melissa González García
AU - Guillaume Vigeral
PY - 2026
DA - 2026/09/16
PB - HAL
DO - 10.48550/arxiv.2609.19905
UR - https://hal.science/hal-05752370v1
AB - We study Percolation Markov Decision Processes (PMDPs), in which the decision maker repeatedly moves a token through the d-dimensional integer lattice with deterministic transitions and random payoffs assigned to the edges. Payoffs are revealed before the beginning of the decision process and the decision maker aims to maximize the average accumulated payoff over a fixed horizon. We establish the existence of the uniform value (as the horizon tends to infinity) and 0-optimal strategies. In the particular case of Bernoulli payoffs, we establish continuity results for the uniform value. We also present several one-dimensional examples to illustrate that optimal strategies may be very complex. Then, we introduce a dimensional lifting property that allows PMDPs to be approximated by PMDPs of lower dimension and could be used to approximate critical probability thresholds in percolation theory.
ER -