The BAR-SOT Method: Long-term Average Cost Control as Stochastic Optimal Self-Transport
Abstract
We reformulate average-cost (ergodic) control of a Markov jump process as a finite-horizon stochastic optimal transport (SOT) problem that jointly optimizes over the controlled evolution and the marginal law from which it starts and returns to (i.e., a self-transport). The constraint relating the marginal flow to the controlled generator is the basic adjoint relationship (BAR), so we call the resulting problem BAR-SOT. For any time horizon T>0, its optimal value is a constant scaling of the long-run average-cost rate. We give three equivalent formulations (through controlled processes, a Fokker--Planck constraint, and relaxed marginal measures) and show that the optimal dual is a stationary potential plus a term linear in time, with slope equal to the rate. A relative-entropy penalty on the control yields a cost-tilted Schrodinger bridge problem, computable by a Sinkhorn-type iteration when every transition rate is controlled, whose value converges to the unregularized optimum as the penalty vanishes. We develop the theory for finite Markov decision processes and then for general controlled Markov jump processes. A neural parametrization of the dual, trained as a physics-informed neural network (PINN), that encodes Harrison's equivalent-workload formulation (see Harrison 2000) matches the strong reinforcement-learning baseline of Dai and Gluzman (2022); numerically conditioning the fit of a sub-dominant transverse correction then improves on both that baseline and the best priority heuristic. On the input-queued switch, a graph-attention parametrization of the dual improves on the strongest matching heuristic we are aware of, with a parameter count that does not grow with the switch size.
Related
- Douglas-Rachford splitting method for nonconvex stochastic optimal control
- Iterative Method for Stochastic Optimal Control Problem Constrained by Parabolic Partial Differential Equation
- Towards Optimal Inventory Control under Censored Demand: A Biased Sample-Average Approximation Approach
- Underdamped Stochastic Optimal Transport with Landauer boundary conditions: machine learning and analytical solution
- Necessary and sufficient conditions for optimal control of coupled forward-backward stochastic control systems driven by G-Brownian motion
BibTeX
@misc{srinivasan2026barsot,
title = {{The BAR-SOT Method: Long-term Average Cost Control as Stochastic Optimal Self-Transport}},
author = {Sharan Srinivasan and Berke M. Turkay and Harsha Honnappa},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2609.17966},
eprint = {2609.17966},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.17966},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - The BAR-SOT Method: Long-term Average Cost Control as Stochastic Optimal Self-Transport
AU - Sharan Srinivasan
AU - Berke M. Turkay
AU - Harsha Honnappa
PY - 2026
DA - 2026/10/23
PB - arXiv
DO - 10.48550/arxiv.2609.17966
UR - https://arxiv.org/abs/2609.17966
AB - We reformulate average-cost (ergodic) control of a Markov jump process as a finite-horizon stochastic optimal transport (SOT) problem that jointly optimizes over the controlled evolution and the marginal law from which it starts and returns to (i.e., a self-transport). The constraint relating the marginal flow to the controlled generator is the basic adjoint relationship (BAR), so we call the resulting problem BAR-SOT. For any time horizon T>0, its optimal value is a constant scaling of the long-run average-cost rate. We give three equivalent formulations (through controlled processes, a Fokker--Planck constraint, and relaxed marginal measures) and show that the optimal dual is a stationary potential plus a term linear in time, with slope equal to the rate. A relative-entropy penalty on the control yields a cost-tilted Schrodinger bridge problem, computable by a Sinkhorn-type iteration when every transition rate is controlled, whose value converges to the unregularized optimum as the penalty vanishes. We develop the theory for finite Markov decision processes and then for general controlled Markov jump processes. A neural parametrization of the dual, trained as a physics-informed neural network (PINN), that encodes Harrison's equivalent-workload formulation (see Harrison 2000) matches the strong reinforcement-learning baseline of Dai and Gluzman (2022); numerically conditioning the fit of a sub-dominant transverse correction then improves on both that baseline and the best priority heuristic. On the input-queued switch, a graph-attention parametrization of the dual improves on the strongest matching heuristic we are aware of, with a parameter count that does not grow with the switch size.
ER -