Intervene, Calibrate, Discover: Learning Causal Structure from Pixels
Abstract
Learning high-level causal variables from low-level observations is a central problem for artificial intelligence. With known node semantics but an unknown graph, visual readout errors can misalign model settings with environment intervention values. We address this mismatch by pairing image readouts with verified values attained through fixed round-robin interventions. With encoder and decoder frozen, a dimension-wise invertible calibration map puts visual responses and target settings in shared numerical units. After freezing this map, the agent selects interventions to resolve graph directions and fits local mechanisms to predict consequences; inverse calibration returns counterfactual states to the decoder's original coordinates. Across Domino, Pick-and-Place, Balance, Gears, CAUSAL3D, and CausalWorld, we evaluate graph discovery, intervention consequences, and counterfactual images. Controlled domino experiments show that interventions resolve orientation ambiguity left by equal additional observations. In a matched-data domino control, calibration reduces the mean error of predicted changes in other variables within the same factual context, relative to an uncalibrated model.
Related
- Which Chains to Calibrate for Physical Backpropagation: Per-User Factors Versus Base-Station Chains
- InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
- Causal Lag Structure Discovery in Confounded Time Series via Orthogonalized Adaptive Estimation
- FOCUS: Fixed-Confidence Online Causal Learning Using Sequential Adaptive Interventions
- Causal Inference with Unobserved Confounding: A Mixture Learning Perspective
BibTeX
@misc{ding2026intervene,
title = {{Intervene, Calibrate, Discover: Learning Causal Structure from Pixels}},
author = {Ziyi Ding and Xiaodi Huang and Xianxin Lai and Haijier Chen and Zhang Zhang and Xiao-Ping Zhang and Jiayu Chen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34711},
eprint = {2610.34711},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34711},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Intervene, Calibrate, Discover: Learning Causal Structure from Pixels
AU - Ziyi Ding
AU - Xiaodi Huang
AU - Xianxin Lai
AU - Haijier Chen
AU - Zhang Zhang
AU - Xiao-Ping Zhang
AU - Jiayu Chen
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.34711
UR - https://arxiv.org/abs/2610.34711
AB - Learning high-level causal variables from low-level observations is a central problem for artificial intelligence. With known node semantics but an unknown graph, visual readout errors can misalign model settings with environment intervention values. We address this mismatch by pairing image readouts with verified values attained through fixed round-robin interventions. With encoder and decoder frozen, a dimension-wise invertible calibration map puts visual responses and target settings in shared numerical units. After freezing this map, the agent selects interventions to resolve graph directions and fits local mechanisms to predict consequences; inverse calibration returns counterfactual states to the decoder's original coordinates. Across Domino, Pick-and-Place, Balance, Gears, CAUSAL3D, and CausalWorld, we evaluate graph discovery, intervention consequences, and counterfactual images. Controlled domino experiments show that interventions resolve orientation ambiguity left by equal additional observations. In a matched-data domino control, calibration reduces the mean error of predicted changes in other variables within the same factual context, relative to an uncalibrated model.
ER -