Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control
Abstract
This monograph provides an introduction to mean field reinforcement learning through the lens of Markov decision processes arising from large-population stochastic control with mean field interactions and common noise. Starting from the connection between multi-agent reinforcement learning and mean field control, it develops the probabilistic, mathematical, and control-theoretic framework needed to formulate representative-agent learning problems, analyze their relationship with finite-population systems, and study both general and linear-quadratic models. The presentation includes dynamic programming principles, propagation-of-chaos limits, and theoretical analyses of tabular Q-learning and policy-gradient methods. It also discusses numerical implementations, including tabular schemes and deep reinforcement learning methods such as deep deterministic policy gradient. The goal is to give readers a coherent bridge between mean field control theory and reinforcement learning methodology, emphasizing the mathematical structure of the problems and the design of tractable learning approaches for large stochastic populations.
Related
- Soft-Update MSA for Nonconvex Stochastic Optimal Control: Theory and Algorithms
- Optimization Frameworks & Learning Algorithms for Discrete-Time Mean Field Games
- Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
- Decentralized Strategies for Finite Population LQG Social Control: A Reinforcement Learning Approach
- Boundary Control of a Modified Timoshenko-Ehrenfest Beam via Reduced-Order Reinforcement Learning
BibTeX
@misc{carmona2026mean,
title = {{Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control}},
author = {René Carmona and Mathieu Laurière},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2607.01525},
eprint = {2607.01525},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.01525},
}
RIS
TY - UNPB
TI - Mean Field Reinforcement Learning: Theory and Algorithms for Mean Field Control
AU - René Carmona
AU - Mathieu Laurière
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2607.01525
UR - https://arxiv.org/abs/2607.01525
AB - This monograph provides an introduction to mean field reinforcement learning through the lens of Markov decision processes arising from large-population stochastic control with mean field interactions and common noise. Starting from the connection between multi-agent reinforcement learning and mean field control, it develops the probabilistic, mathematical, and control-theoretic framework needed to formulate representative-agent learning problems, analyze their relationship with finite-population systems, and study both general and linear-quadratic models. The presentation includes dynamic programming principles, propagation-of-chaos limits, and theoretical analyses of tabular Q-learning and policy-gradient methods. It also discusses numerical implementations, including tabular schemes and deep reinforcement learning methods such as deep deterministic policy gradient. The goal is to give readers a coherent bridge between mean field control theory and reinforcement learning methodology, emphasizing the mathematical structure of the problems and the design of tractable learning approaches for large stochastic populations.
ER -