Optimal sequential decision-making with initiation regimes
Abstract
Consider an optimal dynamic treatment regime, gopt correctly identified from a large, perfectly executed sequentially randomized experiment. Even when the experimental results are generalizable to a future target population, there is no guarantee that gopt outperforms human decision-makers; human experts can do better than gopt whenever they have access to relevant information beyond the covariates recorded in the experiment. Motivated by this observation, we derive results on a new class of regimes called initiation regimes, which generalize existing results on superoptimal regimes. These regimes follow human decision-makers up to the point where it becomes more beneficial to initiate a sequential optimal regime, and are guaranteed to outperform both purely human and purely algorithmic decision rules, e.g., based on reinforcement learning algorithms. Furthermore, we present modified experimental designs that identify the best initiation regimes, show how the best initiation regime can be identified from classical observational data under explicit assumptions, and give estimation and statistical inference methodology for these regimes. To illustrate the practical utility of the methods, we consider initiation regimes in a case study on treatment of lower back pain.
Related
- Optimal Expert Annotation for Evaluating Sequential Decision Policies from Text Records
- Posterior Regimes and Latent Deception: Variational Bayesian Inference in Hidden Markov Models for Sequential Fraud Detection in Financial Transactions
- Classes of Aggregation Rules for Ethical Decision Making in Automated Systems
- Estimating Optimal Dynamic Treatment Regimes for Personalized Education: A Tutorial and Applications with Machine Learning
- Risk-Averse Decision Making with Multi-Level Reliability Guarantees
BibTeX
@misc{laurendeau2026optimal,
title = {{Optimal sequential decision-making with initiation regimes}},
author = {Julien D. Laurendeau and Leora Sarvet and Mats J. Stensrud},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.29844},
eprint = {2609.29844},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.29844},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Optimal sequential decision-making with initiation regimes
AU - Julien D. Laurendeau
AU - Leora Sarvet
AU - Mats J. Stensrud
PY - 2026
DA - 2026/09/24
PB - arXiv
DO - 10.48550/arxiv.2609.29844
UR - https://arxiv.org/abs/2609.29844
AB - Consider an optimal dynamic treatment regime, g^opt correctly identified from a large, perfectly executed sequentially randomized experiment. Even when the experimental results are generalizable to a future target population, there is no guarantee that g^opt outperforms human decision-makers; human experts can do better than g^opt whenever they have access to relevant information beyond the covariates recorded in the experiment. Motivated by this observation, we derive results on a new class of regimes called initiation regimes, which generalize existing results on superoptimal regimes. These regimes follow human decision-makers up to the point where it becomes more beneficial to initiate a sequential optimal regime, and are guaranteed to outperform both purely human and purely algorithmic decision rules, e.g., based on reinforcement learning algorithms. Furthermore, we present modified experimental designs that identify the best initiation regimes, show how the best initiation regime can be identified from classical observational data under explicit assumptions, and give estimation and statistical inference methodology for these regimes. To illustrate the practical utility of the methods, we consider initiation regimes in a case study on treatment of lower back pain.
ER -