InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate
Abstract
Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning. Neural mutual information (MI) estimators offer a promising avenue, but they typically require costly iterative optimization for each new dataset, making them impractical for real-time applications. We present InfoAtlas, a foundation model-like architecture that eliminates this bottleneck by directly inferring MI in a single forward pass. Pretrained on large-scale synthetic data with rich dependence patterns, InfoAtlas learns to identify diverse dependence structures and predict MI directly from the dataset. Comprehensive experiments demonstrate that InfoAtlas matches state-of-the-art neural estimators in accuracy while achieving 100× speedup, can flexibly handle varying dimensions and sample sizes through a single unified model, and generalizes effectively to complex, real-world scenarios. By reformulating MI estimation as an inference task, InfoAtlas establishes a foundation for real-time dependency analysis.
Related
- A zero-density estimate for the symmetric square L-functions in the weight aspect
- Can Tabular Foundation Models Amortize Statistical Inference?
- Uniform Statistical Convergence of Empirical Sinkhorn Potentials with Exponential and Polynomial Dependence on the Regularization Parameter
- HIFI: A Highly Efficient Framework for Statistical Inference with Foundation Models
- A statistical approach to bias in zero-shot learning: the lens of handwriting recognition
BibTeX
@misc{hu2026infoatlas,
title = {{InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate}},
author = {Zhengyang Hu and Yanzhi Chen and Hanxiang Ren and Qunsong Zeng and Youyi Zheng and Adrian Weller and Kaibin Huang and Yanchao Yang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.00241},
eprint = {2606.00241},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.00241},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate
AU - Zhengyang Hu
AU - Yanzhi Chen
AU - Hanxiang Ren
AU - Qunsong Zeng
AU - Youyi Zheng
AU - Adrian Weller
AU - Kaibin Huang
AU - Yanchao Yang
PY - 2026
DA - 2026/09/12
PB - arXiv
DO - 10.48550/arxiv.2606.00241
UR - https://arxiv.org/abs/2606.00241
AB - Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning. Neural mutual information (MI) estimators offer a promising avenue, but they typically require costly iterative optimization for each new dataset, making them impractical for real-time applications. We present InfoAtlas, a foundation model-like architecture that eliminates this bottleneck by directly inferring MI in a single forward pass. Pretrained on large-scale synthetic data with rich dependence patterns, InfoAtlas learns to identify diverse dependence structures and predict MI directly from the dataset. Comprehensive experiments demonstrate that InfoAtlas matches state-of-the-art neural estimators in accuracy while achieving 100× speedup, can flexibly handle varying dimensions and sample sizes through a single unified model, and generalizes effectively to complex, real-world scenarios. By reformulating MI estimation as an inference task, InfoAtlas establishes a foundation for real-time dependency analysis.
ER -