Next Best Action: Causal Constrained Offline RL for Business Action Targeting
Abstract
Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.
Related
- Generative Marketing Mix Modeling: A Causal Inference Framework Linking GEO and GEM to Business Impact
- LASER: Latent Space Adjoint Matching for Support-Constrained Entropy-Regularized Offline RL
- Constraints for Physical Gauge Groups Coming from the Causal Action Principle
- A Geometric Derivation of the Einstein Equations from the Causal Action Principle
- Network-Constrained Matching and Randomization-Based Inference for Causal Effects
BibTeX
@misc{ramachandiran2026next,
title = {{Next Best Action: Causal Constrained Offline RL for Business Action Targeting}},
author = {Surya Ramachandiran and Yiming Li and Juan Mancilla-Caceres and Tao Ye and Jun Liu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.26817},
eprint = {2610.26817},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.26817},
}
RIS
TY - UNPB
TI - Next Best Action: Causal Constrained Offline RL for Business Action Targeting
AU - Surya Ramachandiran
AU - Yiming Li
AU - Juan Mancilla-Caceres
AU - Tao Ye
AU - Jun Liu
PY - 2026
DA - 2026/10/19
PB - arXiv
DO - 10.48550/arxiv.2610.26817
UR - https://arxiv.org/abs/2610.26817
AB - Next Best Action (NBA) systems must select among customer interventions (e.g., discounts, promotions, and lifecycle actions), maximize long-term economic value, satisfy complex operational constraints, and remain causally grounded. Existing approaches lack a rigorous framework for meeting these requirements jointly. We address this gap by formulating NBA as a Constrained Markov Decision Process (CMDP). We then integrate off-policy causal correction of the estimated customer dynamics directly into the Linear Programming (LP) formulation of the CMDP. The resulting policy selects actions based on customer states to maximize long-term value while explicitly enforcing business guardrails. Furthermore, the LP formulation yields shadow prices that quantify the economic trade-offs of operational constraints (e.g., fairness and user welfare), enhancing policy interpretability. We validate our proposed offline Reinforcement Learning (RL) framework on an industry case study involving rideshare coupon targeting, where customer states are defined by Recency, Frequency, and Monetary (RFM) stages. We evaluate performance using model-based off-policy evaluation, with causal dynamics estimated from a temporal holdout. Under the observed coupon budget and RFM state representation, our optimized policy improves customer lifetime value (LTV) by 19% over the estimated behavior policy and 8% over a myopic baseline. Although the demonstrated action space is binary, the proposed framework is action-agnostic, yielding a general-purpose decision module for targeting systems that manage multiple interventions.
ER -