Can Tabular Foundation Models Amortize Statistical Inference?
Abstract
For decades, statistical inference has largely been developed one problem at a time. Given a scientific target, such as a treatment effect or a regression function, statisticians design a problem-specific estimator together with a procedure for quantifying its uncertainty. This paper proposes a different paradigm. We focus on a classical problem in statistical inference, confidence interval construction, and develop TabCon, an amortized inference system built on a tabular foundation model that produces confidence intervals for new datasets through a simple forward pass. The key methodological ingredients of TabCon are a sparse mixture-of-experts architecture and reinforcement-learning-based post-training that calibrate the resulting confidence intervals to a desired coverage level. Across a wide range of benchmark datasets, TabCon attains near-nominal coverage while producing short confidence intervals. At inference time, it also offers considerably greater computational efficiency, running 50 times faster than the classical bootstrap procedure, even when the latter uses only 50 bootstrap samples.
Related
- HIFI: A Highly Efficient Framework for Statistical Inference with Foundation Models
- How Well Do Prior-Data Fitted Networks Amortize Bayesian Inference?
- Conformal Prediction and Conditional Coverage for Tabular Foundation Models
- SurvFM enables tabular foundation models for right-censored survival prediction
- Estimating Uncoded Crash Factors with Tabular Foundation and System One Models: Kumo Tabular and Jev
BibTeX
@misc{ye2026tabular,
title = {{Can Tabular Foundation Models Amortize Statistical Inference?}},
author = {Kai Ye and Shijin Gong and Hongyi Zhou and Valentina Zangirolami and Chengchun Shi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.33114},
eprint = {2609.33114},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.33114},
}
RIS
TY - UNPB
TI - Can Tabular Foundation Models Amortize Statistical Inference?
AU - Kai Ye
AU - Shijin Gong
AU - Hongyi Zhou
AU - Valentina Zangirolami
AU - Chengchun Shi
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.33114
UR - https://arxiv.org/abs/2609.33114
AB - For decades, statistical inference has largely been developed one problem at a time. Given a scientific target, such as a treatment effect or a regression function, statisticians design a problem-specific estimator together with a procedure for quantifying its uncertainty. This paper proposes a different paradigm. We focus on a classical problem in statistical inference, confidence interval construction, and develop TabCon, an amortized inference system built on a tabular foundation model that produces confidence intervals for new datasets through a simple forward pass. The key methodological ingredients of TabCon are a sparse mixture-of-experts architecture and reinforcement-learning-based post-training that calibrate the resulting confidence intervals to a desired coverage level. Across a wide range of benchmark datasets, TabCon attains near-nominal coverage while producing short confidence intervals. At inference time, it also offers considerably greater computational efficiency, running 50 times faster than the classical bootstrap procedure, even when the latter uses only 50 bootstrap samples.
ER -