InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
Abstract
Multimodal large language models often capture visual-linguistic correlations but struggle to predict how local visual interventions propagate and affect downstream answers. We introduce InfluenceField, an intervention-aware latent field inserted between the visual encoder and language decoder. It lifts patch features into a continuous spatial representation, propagates directed influence over multiple steps, and predicts local intervention effects through a shared transition operator. Training jointly optimizes language modeling, cross-environment invariance, counterfactual rollout supervision, and structural regularization. For a nonlinear finite-basis population model, we show that target-aligned interventional supervision, together with a one-step separation condition on the transition, restricts admissible representations to within-location reparameterizations, so that the directed dependency graph of the full transition is recovered exactly. A linear specialization gives an exact partial-coverage characterization and a finite-loss stability bound, and the field analysis derives the spatial profile of coefficient interventions together with a shared-channel calibration result. On CausalVQA, InfluenceField improves overall accuracy over its backbone by 13.1 percentage points, with the largest gains on the planning and hypothetical categories. Capacity-matched baselines and structural controls attribute the gains in robustness and factual-counterfactual consistency to the causal objectives rather than to added capacity.
Related
- DeepTelecom: A Differentiable Multimodal Framework for Channel Twinning in Dynamic Heterogeneous Wireless Environments
- Polynomial Residual Networks are Generically Identifiable
- Identifiable Uniqueness of Impulsively Forced, Inert, and Stabilized Body Trajectories
- An Unbounded Gap Between Identification and Identifiable Elicitation: Central Moments
- Weak Limits of Parameter Estimation in Strongly Identifiable Mixture Models
BibTeX
@misc{yang2026influencefield,
title = {{InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling}},
author = {Zihao Yang and Zijia Wang and Zhiqiu Huang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.07874},
eprint = {2609.07874},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.07874},
}
RIS
TY - UNPB
TI - InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
AU - Zihao Yang
AU - Zijia Wang
AU - Zhiqiu Huang
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2609.07874
UR - https://arxiv.org/abs/2609.07874
AB - Multimodal large language models often capture visual-linguistic correlations but struggle to predict how local visual interventions propagate and affect downstream answers. We introduce InfluenceField, an intervention-aware latent field inserted between the visual encoder and language decoder. It lifts patch features into a continuous spatial representation, propagates directed influence over multiple steps, and predicts local intervention effects through a shared transition operator. Training jointly optimizes language modeling, cross-environment invariance, counterfactual rollout supervision, and structural regularization. For a nonlinear finite-basis population model, we show that target-aligned interventional supervision, together with a one-step separation condition on the transition, restricts admissible representations to within-location reparameterizations, so that the directed dependency graph of the full transition is recovered exactly. A linear specialization gives an exact partial-coverage characterization and a finite-loss stability bound, and the field analysis derives the spatial profile of coefficient interventions together with a shared-channel calibration result. On CausalVQA, InfluenceField improves overall accuracy over its backbone by 13.1 percentage points, with the largest gains on the planning and hypothetical categories. Capacity-matched baselines and structural controls attribute the gains in robustness and factual-counterfactual consistency to the causal objectives rather than to added capacity.
ER -