Treatment Effect Modification by Latent Classes
Abstract
Many treatments have effects that plausibly vary by latent classes. For example, the effect of personalized video recommendations may be stronger for users who seek a new discovery than for users seeking to rewatch something familiar. In the context of user journeys on digital platforms, we can conceptualize user moods as dynamic latent states that lead to distinct action distributions. Because mood is unobserved, we infer it from a user's pre-treatment actions using a hidden Markov model (HMM), and then leverage the latent state as an interpretable treatment effect modifier when evaluating experiment results. Under a treatment-effect sufficiency assumption, each observable conditional effect is a known mixture of the latent effects, so state-specific effects are identified when state posteriors are sufficiently heterogeneous. We propose an estimator that is Neyman-orthogonal to errors in the fitted HMM, enabling standard influence-function confidence intervals. In a real-world large-scale A/B test of a new recommendation model at Netflix, our latent-state analysis reveals that heavy-engagement users show significantly positive effects across all treatment variants, with limited evidence of benefit among less-active and rewatch-oriented users.
Related
- Local average treatment effects on latent outcomes
- Identifying adverse event clusters and treatment effects in randomised trials using latent class analysis: an application in the COV-BOOST trial
- Splitting Brauer classes by genus one curves over number fields
- Classes of Aggregation Rules for Ethical Decision Making in Automated Systems
- Functorial equivalence classes of 2-blocks of tame representation type
BibTeX
@misc{lan2026treatment,
title = {{Treatment Effect Modification by Latent Classes}},
author = {Hui Lan and Nathan Kallus and Winston Chou},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.10851},
eprint = {2610.10851},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10851},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Treatment Effect Modification by Latent Classes
AU - Hui Lan
AU - Nathan Kallus
AU - Winston Chou
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.10851
UR - https://arxiv.org/abs/2610.10851
AB - Many treatments have effects that plausibly vary by latent classes. For example, the effect of personalized video recommendations may be stronger for users who seek a new discovery than for users seeking to rewatch something familiar. In the context of user journeys on digital platforms, we can conceptualize user moods as dynamic latent states that lead to distinct action distributions. Because mood is unobserved, we infer it from a user's pre-treatment actions using a hidden Markov model (HMM), and then leverage the latent state as an interpretable treatment effect modifier when evaluating experiment results. Under a treatment-effect sufficiency assumption, each observable conditional effect is a known mixture of the latent effects, so state-specific effects are identified when state posteriors are sufficiently heterogeneous. We propose an estimator that is Neyman-orthogonal to errors in the fitted HMM, enabling standard influence-function confidence intervals. In a real-world large-scale A/B test of a new recommendation model at Netflix, our latent-state analysis reveals that heavy-engagement users show significantly positive effects across all treatment variants, with limited evidence of benefit among less-active and rewatch-oriented users.
ER -