{
  "abstract": "This paper develops a model-free framework for continuous-time mean-field control when the population evolves according to unknown controlled McKean--Vlasov dynamics and only discrete-time transition data are available. Model-based mean-field control requires the continuous-time drift and diffusion coefficients, which are not directly observed from fixed-step transitions, while a direct reduction to a discrete-time Bellman equation loses the continuous-time generator structure. To bridge these two viewpoints, we introduce a Mean-Field-PhiBE (MF-PhiBE), which incorporates discrete-time transition information into a continuous-time PDE on the Wasserstein space. The MF-PhiBE replaces the unknown infinitesimal drift and covariance in the policy-evaluation equation by one-step estimators computed from data, while preserving the generator structure of the McKean-Vlasov HJB equation. We also derive a policy-gradient theorem for entropy-regularized randomized feedback policies, expressing the actor direction through an action-wise infinitesimal advantage and the score of the policy. Combining these two ingredients yields a model-free actor-critic method. We prove a first-order consistency estimate showing that the value induced by an optimal MF-PhiBE policy approximates the optimal continuous-time value as the observation time step vanishes. For entropy-regularized LQR, we establish first-order policy convergence and second-order value convergence; under suitable conditions, the population-averaged feedback means coincide exactly. Numerical experiments on an LQR benchmark and a crowd-aversion problem illustrate the proposed framework.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2606.26498",
  "author_total": 4,
  "authors": [
    {
      "name": "Erhan Bayraktar"
    },
    {
      "name": "Martin Hernandez"
    },
    {
      "name": "Qinxin Yan"
    },
    {
      "name": "Yuhua Zhu"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/9hsdnzw31jmrr.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2606.26498",
  "fulltext_url": null,
  "id": "9hsdnzw31jmrr",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2606.26498",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "math.OC"
    ],
    [
      "openalex",
      "2003|Stochastic processes and financial applications"
    ]
  ],
  "openalex": "W7166048858",
  "pdf_url": "https://arxiv.org/pdf/2606.26498",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-05",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/9hsdnzw31jmrr.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:19:57Z",
      "native_id": "2606.26498",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2606.26498"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:math.OC",
      "subfield": "optimisation-control"
    }
  ],
  "title": "Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data",
  "title_text": "Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data",
  "type": "preprint",
  "updated_at": "2026-10-30T12:19:57Z",
  "url": "https://research.draconitic.com/paper/9hsdnzw31jmrr",
  "venue": "arXiv",
  "versions": []
}