{
  "abstract": "We study decentralized learning of Nash equilibria (NE) in infinite-horizon discounted Markov games under bandit feedback, focusing on Markov $α$-potential games. We develop KL-projected natural policy gradient (NPG) algorithms in two settings: an episodic setting with frozen policies during sampling and a fully online setting in which players receive a single realized cost sample per time step and update their policies asynchronously along a continuing trajectory. We establish finite-time high-probability NE regret bounds of order $\\widetilde O(T^{-1/4})$ and $\\widetilde O(T^{-2/15})$ for the episodic and fully online settings, respectively, up to fixed approximation terms. Crucially, our bounds eliminate the distribution-mismatch coefficient, which can scale prohibitively with the size of the state space, while accommodating potential approximation, estimation-oracle bias, and transition sensitivity. We further identify a state-wise potential structure that yields sharper guarantees with additive dependence on the potential approximation error $α$. We specialize the framework to independent-resource Markov congestion games (IMCGs), establish their approximate-potential and transition-sensitivity properties, and construct decentralized estimation oracles from realized costs. As an application, we introduce strategic online job scheduling on stochastic machines and obtain a scalable decentralized algorithm for learning stable dispatching policies. Overall, our results provide the first finite-time high-probability NE regret guarantees for fully online asynchronous decentralized learning in Markov $α$-potential games, remove distribution-mismatch coefficients from the regret bounds, accommodate fixed estimation-oracle bias, and provide scalable decentralized learning with finite-time guarantees for IMCGs.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2609.14959",
  "author_total": 1,
  "authors": [
    {
      "name": "S. Rasoul Etesami"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/brnjf30yfaj69.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2609.14959",
  "fulltext_url": null,
  "id": "brnjf30yfaj69",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2609.14959",
  "language": null,
  "licence": "LicenseRef-arXiv-nonexclusive-1.0",
  "licence_label": "arXiv licence",
  "licence_source": "article",
  "licence_url": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.GT"
    ],
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "cs.MA"
    ],
    [
      "arxiv",
      "cs.SY"
    ],
    [
      "arxiv",
      "eess.SY"
    ],
    [
      "arxiv",
      "math.OC"
    ],
    [
      "openalex",
      "1803|Advanced Bandit Algorithms Research"
    ]
  ],
  "openalex": "W7213351741",
  "pdf_url": "https://arxiv.org/pdf/2609.14959",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-14",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/brnjf30yfaj69.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:21:29Z",
      "native_id": "2609.14959",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2609.14959"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:math.OC",
      "subfield": "optimisation-control"
    }
  ],
  "title": "High-Probability Nash Regret for Decentralized Learning in Markov $α$-Potential Games: Episodic and Fully Online Asynchronous Algorithms with Applications to Markov Congestion Games",
  "title_text": "High-Probability Nash Regret for Decentralized Learning in Markov α-Potential Games: Episodic and Fully Online Asynchronous Algorithms with Applications to Markov Congestion Games",
  "type": "preprint",
  "updated_at": "2026-10-30T12:21:29Z",
  "url": "https://research.draconitic.com/paper/brnjf30yfaj69",
  "venue": "arXiv",
  "versions": []
}