Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies
Abstract
We study entropy-regularized exploratory control in finite N-player stochastic differential games under a response model in which each player conditions on the opponents' currently realized actions and evaluates continuation with that profile frozen. The resulting Gibbs best responses form a system of full conditional densities, which need not admit a common joint law. We characterize joint realizability by a cross-partial condition on the entropy-scaled Hamiltonian gradients and, on simply connected action domains, by an equivalent entropy-weighted potential structure. When compatibility fails, a coordinate-path construction yields a joint density whose full conditionals satisfy explicit quadratic Kullback--Leibler bounds. We extend the analysis to stationary discounted problems and derive martingale and policy-improvement characterizations for learning the frozen response maps. A two-player linear-quadratic example illustrates the compatibility criterion and the associated learning procedure.
Related
- Continuous Delayed-Memory Stochastic Gradient Descent and Continuous-Time Reinforcement Learning from History of Astrophysical Time Series Studies
- Reinforcement learning for constrained stochastic linear--quadratic control in continuous time
- Model-free Reinforcement Learning for Continuous Time and State: A Stochastic Maximum Principle Approach
- Pathwise uniqueness for degenerate stochastic differential equations with Hölder continuous coefficients
- Multidimensional Infinite-Horizon Markovian Backward Stochastic Differential Equations with Continuous Coefficients and Nonzero-Sum Differential Games
BibTeX
@misc{liu2026continuoustime,
title = {{Continuous-Time Reinforcement Learning for $N$-Player Stochastic Differential Games with Exploratory Policies}},
author = {Jisheng Liu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.19928},
eprint = {2607.19928},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.19928},
}
RIS
TY - UNPB
TI - Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies
AU - Jisheng Liu
PY - 2026
DA - 2026/09/19
PB - arXiv
DO - 10.48550/arxiv.2607.19928
UR - https://arxiv.org/abs/2607.19928
AB - We study entropy-regularized exploratory control in finite N-player stochastic differential games under a response model in which each player conditions on the opponents' currently realized actions and evaluates continuation with that profile frozen. The resulting Gibbs best responses form a system of full conditional densities, which need not admit a common joint law. We characterize joint realizability by a cross-partial condition on the entropy-scaled Hamiltonian gradients and, on simply connected action domains, by an equivalent entropy-weighted potential structure. When compatibility fails, a coordinate-path construction yields a joint density whose full conditionals satisfy explicit quadratic Kullback--Leibler bounds. We extend the analysis to stationary discounted problems and derive martingale and policy-improvement characterizations for learning the frozen response maps. A two-player linear-quadratic example illustrates the compatibility criterion and the associated learning procedure.
ER -