Skip to content
DraconiticResearch
Search Sources API
197,768 studies31 sourcesUpdated 1 h ago
PreprintarXiv licencearXiv

Causal Foundation Models

  • Christopher Stith
  • Hossein Rahmani
  • Jesse C. Cresswell
Open at sourcePDFDOICite

Abstract

Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.

Related

  1. Raw-Routed Mixture of Adapters: A Causal Intervention for Routing Collapse in Time Series Foundation Models
    Hung Phan, Thuy T. Nguyen, Minh Ngoc Dinh +1
    arXivPreprintarXiv licenceFull textStatisticsarXiv
  2. Unit-level aggregation breaks coverage in causal foundation models
    Gabe Schulman, Lucius E. J. Bynum, Yoav Wald
    arXivPreprintCC BY 4.0Full textStatisticsarXiv
  3. What masking geometry works best for EEG foundation models?
    Pierre Guetschel, Bruno Aristimunha, Yassine El Ouahidi +3
    HALPreprintLicence not statedNeuroscienceStatisticsHAL
  4. Limits and promises of earth observation foundation models in predicting multi-trophic soil biodiversity
    Selene Cerna, Sara Si-Moussi, Irene Calderón-Sanou +2
    Advances in Ecological ResearchPeer-reviewedLicence not statedFull text1 citedEnvironmental science & pollutionStatisticsHAL
  5. Existence of bases implies the axiom of choice, a foundation-free proof
    Gabriel Fernandes, Renan Maneli Mezabarba, Vinicius de Oliveira Rodrigues
    arXivPreprintCC BY 4.0Full textLogic & foundationsarXiv

Cite

BibTeXRIS
BibTeX
@misc{stith2026causal,
  title = {{Causal Foundation Models}},
  author = {Christopher Stith and Hossein Rahmani and Jesse C. Cresswell},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.03003},
  eprint = {2609.03003},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.03003},
}
RIS
TY  - UNPB
TI  - Causal Foundation Models
AU  - Christopher Stith
AU  - Hossein Rahmani
AU  - Jesse C. Cresswell
PY  - 2026
DA  - 2026/09/02
PB  - arXiv
DO  - 10.48550/arxiv.2609.03003
UR  - https://arxiv.org/abs/2609.03003
AB  - Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.
ER  -