Continuous-time q-Learning for Finite-Horizon Portfolio Choice with Optimal Stopping in Jump-Diffusion Markets
Abstract
We develop a structured continuous-time q-learning framework for finite-horizon investment and optimal stopping under jump-diffusion dynamics with unknown parameters. A penalty approximation transforms the Hamilton-Jacobi-Bellman variational inequality into an HJB equation with a binary control for the stopping penalty. An auxiliary state incorporates the resulting control-dependent discount factor into a Markovian formulation. We randomize controls on the mixed action space and introduce Shannon entropy regularization, establishing policy improvement and martingale characterizations of the continuous-time q-function. These results motivate a structured q-learning algorithm that constructs the q-function from the penalty HJB equation and jointly learns the value function and unknown generator parameters from sample paths. We apply the method to portfolio choice with optimal stopping in a simulation study. We further conduct a two-asset empirical study, evaluating learned strategies on bootstrap paths from held-out test returns versus an MLE-finite difference benchmark.
Related
- Continuous-Time Mean-Variance Portfolio Selection Reinforcement Learning Problems for a Markov Regime-Switching System with No-Shorting Constraints
- Optimal Transport for Continuous-Time Mean-Variance Portfolio Selection
- A Legendre-Based Modified Extreme Learning Machine for Convection-Diffusion Equations: Steady-State, Continuous-Time, and Discrete-Time Formulations
- Diffusion under competing bulk and surface stopping mechanisms
- Minimax-Optimal Early Stopping for Continuous-Time SGD via the Discrepancy Principle
BibTeX
@misc{zhang2026continuoustime,
title = {{Continuous-time q-Learning for Finite-Horizon Portfolio Choice with Optimal Stopping in Jump-Diffusion Markets}},
author = {Tingting Zhang and Can Yang},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.07362},
eprint = {2611.07362},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.07362},
}
RIS
TY - UNPB
TI - Continuous-time q-Learning for Finite-Horizon Portfolio Choice with Optimal Stopping in Jump-Diffusion Markets
AU - Tingting Zhang
AU - Can Yang
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.07362
UR - https://arxiv.org/abs/2611.07362
AB - We develop a structured continuous-time q-learning framework for finite-horizon investment and optimal stopping under jump-diffusion dynamics with unknown parameters. A penalty approximation transforms the Hamilton-Jacobi-Bellman variational inequality into an HJB equation with a binary control for the stopping penalty. An auxiliary state incorporates the resulting control-dependent discount factor into a Markovian formulation. We randomize controls on the mixed action space and introduce Shannon entropy regularization, establishing policy improvement and martingale characterizations of the continuous-time q-function. These results motivate a structured q-learning algorithm that constructs the q-function from the penalty HJB equation and jointly learns the value function and unknown generator parameters from sample paths. We apply the method to portfolio choice with optimal stopping in a simulation study. We further conduct a two-asset empirical study, evaluating learned strategies on bootstrap paths from held-out test returns versus an MLE-finite difference benchmark.
ER -