{
  "abstract": "We study reinforcement learning (RL) with transition look-ahead, where the agent may observe which states would be visited upon playing any sequence of actions before deciding its course of action. Although look-ahead can substantially improve achievable performance, [1] showed that optimal planning with multi-step transition look-ahead is NP-hard. However, this hardness was established using a discount factor close to one. It was therefore unknown whether the problem remains hard for every discount factor, and whether near-optimal planning can nevertheless be performed efficiently. We resolve both questions. First, we show that for every fixed discount factor, exact planning remains NP-hard. Second, we introduce a randomized polynomial-time approximation scheme for every fixed look-ahead depth. Third, we extend our approach to account for unknown transitions. We empirically validate the soundness of our results on the wind-farm storage-control benchmark of [2], showing that our approach, optimally accounting for -step look-ahead information, offers substantially better performance than existing algorithms. [1] Corentin Pla, Hugo Richard, Marc Abeille, Nadav Merlis, Vianney Perchet : On the Hardness of Reinforcement Learning with Transition Look-Ahead [2] Chenbei Lu, Zaiwei Chen, Tongxin Li, Chenye Wu, Adam Wierman : Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2609.11807",
  "author_total": 4,
  "authors": [
    {
      "name": "Corentin Pla"
    },
    {
      "name": "Hugo Richard"
    },
    {
      "name": "Marc Abeille"
    },
    {
      "name": "Vianney Perchet"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/7p6f93t6zmzh2.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2609.11807",
  "fulltext_url": null,
  "id": "7p6f93t6zmzh2",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2609.11807",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "stat.ML"
    ],
    [
      "openalex",
      "1702|Reinforcement Learning in Robotics"
    ]
  ],
  "openalex": "W7212239458",
  "pdf_url": "https://arxiv.org/pdf/2609.11807",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-28",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/7p6f93t6zmzh2.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:25:26Z",
      "native_id": "2609.11807",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2609.11807"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:stat.ML",
      "subfield": "statistics"
    }
  ],
  "title": "Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead",
  "title_text": "Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead",
  "type": "preprint",
  "updated_at": "2026-10-30T12:25:26Z",
  "url": "https://research.draconitic.com/paper/7p6f93t6zmzh2",
  "venue": "arXiv",
  "versions": []
}