Are Good Generators Good Decision-Makers? Policy Learning for General Interventions via Retargeted Counterfactual Generation
Abstract
Generative models are increasingly used to support decision-making in complex systems, where interventions may be joint and high-dimensional, and outcomes are high-dimensional. However, using generators for these decision-making settings are challenged by three problems. First, they are often trained on noisy logs with limited intervention data. Second, generator learning can be noisy across different environments. Third, generators are not optimized for the decisions they support. We introduce policy learning via retargeted counterfactual generation, which trains a generator for the decisions it supports in three steps. We (1) learn a doubly-robust, invariant counterfactual generator for high-dimensional interventions and outcomes, (2) conduct policy-learning based on its rollouts, and (3) retarget the generator toward the learned policy's interventions and relearn the policy, so the generator is accurate where decisions are made. Theoretically, our generator's excess counterfactual risk has a doubly robust product remainder, and retargeting removes the worst-case density ratio between the logged and learned policies from the regret. We demonstrate the efficacy of our approach across synthetic data, Cell Painting images of SARS-CoV-2-infected cells, and physical video simulations.
Related
- Decision-Tree Surrogates for Weak Counterfactual Explanations in Integer Optimization
- An integrated explainable AI framework with counterfactual optimization for decision support
- Counterfactual Generation via Flow Matching: Coupling-Sensitive End-to-End Rates
- UQ-ACE: Stability-Aware Counterfactual Controller Tuning via Bayesian SINDy
- A Counterfactual Framework for Estimating Infectious Disease Prevalence under Repeated Testing with Symptomatic and Contact-Tracing Components
BibTeX
@misc{kim2026good,
title = {{Are Good Generators Good Decision-Makers? Policy Learning for General Interventions via Retargeted Counterfactual Generation}},
author = {Raphael C Kim and Jingsen Zhu and Ramin Zabih and Michele Santacatterina},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2606.07399},
eprint = {2606.07399},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.07399},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Are Good Generators Good Decision-Makers? Policy Learning for General Interventions via Retargeted Counterfactual Generation
AU - Raphael C Kim
AU - Jingsen Zhu
AU - Ramin Zabih
AU - Michele Santacatterina
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2606.07399
UR - https://arxiv.org/abs/2606.07399
AB - Generative models are increasingly used to support decision-making in complex systems, where interventions may be joint and high-dimensional, and outcomes are high-dimensional. However, using generators for these decision-making settings are challenged by three problems. First, they are often trained on noisy logs with limited intervention data. Second, generator learning can be noisy across different environments. Third, generators are not optimized for the decisions they support. We introduce policy learning via retargeted counterfactual generation, which trains a generator for the decisions it supports in three steps. We (1) learn a doubly-robust, invariant counterfactual generator for high-dimensional interventions and outcomes, (2) conduct policy-learning based on its rollouts, and (3) retarget the generator toward the learned policy's interventions and relearn the policy, so the generator is accurate where decisions are made. Theoretically, our generator's excess counterfactual risk has a doubly robust product remainder, and retargeting removes the worst-case density ratio between the logged and learned policies from the regret. We demonstrate the efficacy of our approach across synthetic data, Cell Painting images of SARS-CoV-2-infected cells, and physical video simulations.
ER -