Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning
Abstract
We study stationary discrete-time mean-field games with finite state and action spaces under the average-reward criterion, with an entropy-regularized objective and population-dependent dynamics and rewards. Under a minorization condition, the soft Q-operator is a span-seminorm contraction for the full transition kernel, so relative value iteration does not require the minorizing measure. When the Lipschitz constant of the kernel is below the minorization mass and the temperature exceeds an explicit threshold, the equilibrium fixed-point operator is a contraction: an equilibrium exists, its population distribution is unique, and a fixed-point iteration converges to it geometrically. We then learn the equilibrium model-free from a simulator, combining soft Q-learning at a fixed population distribution with an empirical transition kernel. We prove a finite-sample bound for soft Q-learning in the span seminorm, with temperature-independent constants, and derive convergence rates for the learned population distribution and policy. A congestion-aware routing game illustrates the results.
Related
- Entropy-Regularized Reinforcement Learning for the Regime-Switching Consumption-Investment Problem
- Entropy-Regularized Optimal Transport for Time-Varying Multi-Agent Coverage Control
- Accelerating Sinkhorn for Entropy-Regularized Optimal Transport
- Entropy-Regularized Finite-State Mean Field Control: Regularity and Stability
- Partial identification with entropy regularized optimal transport
BibTeX
@misc{akbas2026entropyregularized,
title = {{Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning}},
author = {Gunes Akbas and Naci Saldi and Berkay Anahtarci and Can Deha Kariksiz},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.15871},
eprint = {2610.15871},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.15871},
}
RIS
TY - UNPB
TI - Entropy-Regularized Mean-Field Games with Average-Reward Criterion: Span-Seminorm Contractions and Model-Free Learning
AU - Gunes Akbas
AU - Naci Saldi
AU - Berkay Anahtarci
AU - Can Deha Kariksiz
PY - 2026
DA - 2026/10/11
PB - arXiv
DO - 10.48550/arxiv.2610.15871
UR - https://arxiv.org/abs/2610.15871
AB - We study stationary discrete-time mean-field games with finite state and action spaces under the average-reward criterion, with an entropy-regularized objective and population-dependent dynamics and rewards. Under a minorization condition, the soft Q-operator is a span-seminorm contraction for the full transition kernel, so relative value iteration does not require the minorizing measure. When the Lipschitz constant of the kernel is below the minorization mass and the temperature exceeds an explicit threshold, the equilibrium fixed-point operator is a contraction: an equilibrium exists, its population distribution is unique, and a fixed-point iteration converges to it geometrically. We then learn the equilibrium model-free from a simulator, combining soft Q-learning at a fixed population distribution with an empirical transition kernel. We prove a finite-sample bound for soft Q-learning in the span seminorm, with temperature-independent constants, and derive convergence rates for the learned population distribution and policy. A congestion-aware routing game illustrates the results.
ER -