Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions

Abstract

High-frequency control problems are often studied through time discretization, but refining the timestep δ increases the discrete horizon H=T/δ and can make standard finite-horizon regret guarantees deteriorate even when the physical horizon T is fixed. We study episodic value-based reinforcement learning for Euler discretizations of controlled diffusions. The performance is measured as the cumulative regret over K episodes relative to the optimal policy of the corresponding Euler model. Under stability and regularity conditions on value function gradients, Bellman residual derivatives, and fitted critics, we prove an 𝒪(log K) regret bound with constants uniform in δ. For dynamics with drift linear in unknown parameters and unknown constant diffusion covariance, we verify localized versions of these conditions for a constrained, model-based controlled-diffusion fitted-Q algorithm (CD-FQ) and obtain high-probability 𝒪(log Kloglog K) regret for fixed and finite T, again with constants uniform in δ.

Tighter Regret Bounds for Contextual Action-Set Reinforcement Learning
Zijun Chen, Zihan Zhang
arXiv30 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv
Minimax Regret for Sobolev Kernel-Based Reinforcement Learning
Haichen Hu, David Simchi-Levi
arXiv25 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv
Online tensor learning: Computational and statistical trade-offs, adaptivity and optimal regret
Jingyang Li, Jian‐Feng Cai, Yang Chen +1
The Annals of Statistics1 Jun 2026Peer-reviewedLicence not statedFull text16 citedNumerical analysis & scientific computingOpenAlex
Uniform boundary observability for periodic wave equations with low frequency exponent 4/3
Zhonggan Huang, Jinping Zhuge
arXiv3 Nov 2026PreprintarXiv licenceFull textAnalysis & PDEsarXiv
Regret Lower Bound for Online Learning of Quantum States with Logarithmic Loss, with Applications to Quantum State Tomography and Confidence Sequences for Matrix Means
Yen-Huan Li, Wei-Fu Tseng
arXiv15 Oct 2026PreprintarXiv licenceFull textStatistics Quantum physics & quantum informationarXiv

Cite

BibTeX RIS

BibTeX

@misc{blanchet2026frequencyuniform,
  title = {{Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions}},
  author = {Jose Blanchet and Lutong Hao and Renyuan Xu},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.36758},
  eprint = {2610.36758},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.36758},
}

RIS

TY  - UNPB
TI  - Frequency-Uniform Log-Regret Reinforcement Learning for Diffusions
AU  - Jose Blanchet
AU  - Lutong Hao
AU  - Renyuan Xu
PY  - 2026
DA  - 2026/10/27
PB  - arXiv
DO  - 10.48550/arxiv.2610.36758
UR  - https://arxiv.org/abs/2610.36758
AB  - High-frequency control problems are often studied through time discretization, but refining the timestep δ increases the discrete horizon H=T/δ and can make standard finite-horizon regret guarantees deteriorate even when the physical horizon T is fixed. We study episodic value-based reinforcement learning for Euler discretizations of controlled diffusions. The performance is measured as the cumulative regret over K episodes relative to the optimal policy of the corresponding Euler model. Under stability and regularity conditions on value function gradients, Bellman residual derivatives, and fitted critics, we prove an 𝒪(log K) regret bound with constants uniform in δ. For dynamics with drift linear in unknown parameters and unknown constant diffusion covariance, we verify localized versions of these conditions for a constrained, model-based controlled-diffusion fitted-Q algorithm (CD-FQ) and obtain high-probability 𝒪(log Kloglog K) regret for fixed and finite T, again with constants uniform in δ.
ER  -

Abstract

Related

Cite