Augmented Patient Preference Incorporated Reinforcement Learning (APP-RL) to Estimate the Optimal Dynamic Treatment Regime
Abstract
Dynamic treatment regimes (DTRs) are sequential decision rules that individualize treatments to each patient at each treatment stage adapting to their past clinical course. Existing literature typically accommodates each individual's medical history, but overlooks a patient's preferences. We propose a method that incorporates a patient's latent preferences through data augmentation into a tree-based reinforcement learning method to estimate optimal dynamic treatment regimes for multi-stage, multi-treatment settings. For each patient at each stage, we derive the posterior distribution of preferences given responses to a questionnaire, and then subsequently weight multiple outcomes with the estimated preferences to identify the optimal stage-wise personalized decision. For multiple stage situations, we grow a decision tree at each stage and implement the algorithm recursively using backward induction. Our proposed method, named Augmented Patient Preference incorporated Reinforcement Learning (APP-RL) is robust, efficient, and leads to interpretable DTR estimation. The finite-sample performances of the proposed method has been thoroughly evaluated through simulation studies.
Related
- Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes
- Confounding Masquerading as Improvement: A Systematic Evaluation of Offline Reinforcement Learning for Stroke Antithrombotic Treatment in a 129,000-Patient Registry
- Chance-Constrained Personalized LLM Routing with Online Preference Learning
- Reliability-Weighted Direct Preference Optimization: Penalized Weights for Robust Preference Learning
- PREFER: Personalized Review Summarization with Online Preference Learning
BibTeX
@misc{zhong2026augmented,
title = {{Augmented Patient Preference Incorporated Reinforcement Learning (APP-RL) to Estimate the Optimal Dynamic Treatment Regime}},
author = {Yingchao Zhong and Lu Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.01044},
eprint = {2610.01044},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.01044},
}
RIS
TY - UNPB
TI - Augmented Patient Preference Incorporated Reinforcement Learning (APP-RL) to Estimate the Optimal Dynamic Treatment Regime
AU - Yingchao Zhong
AU - Lu Wang
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.01044
UR - https://arxiv.org/abs/2610.01044
AB - Dynamic treatment regimes (DTRs) are sequential decision rules that individualize treatments to each patient at each treatment stage adapting to their past clinical course. Existing literature typically accommodates each individual's medical history, but overlooks a patient's preferences. We propose a method that incorporates a patient's latent preferences through data augmentation into a tree-based reinforcement learning method to estimate optimal dynamic treatment regimes for multi-stage, multi-treatment settings. For each patient at each stage, we derive the posterior distribution of preferences given responses to a questionnaire, and then subsequently weight multiple outcomes with the estimated preferences to identify the optimal stage-wise personalized decision. For multiple stage situations, we grow a decision tree at each stage and implement the algorithm recursively using backward induction. Our proposed method, named Augmented Patient Preference incorporated Reinforcement Learning (APP-RL) is robust, efficient, and leads to interpretable DTR estimation. The finite-sample performances of the proposed method has been thoroughly evaluated through simulation studies.
ER -