Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression
Abstract
Conditional density estimation (CDE) -recovering the full conditional distribution of a response given tabular covariates -is essential in settings with heteroscedasticity, multimodality, or asymmetric uncertainty. Recent tabular foundation models, such as TabPFN and TabICL, naturally produce predictive distributions, but their effectiveness as generalpurpose CDE methods has not been systematically evaluated -unlike their performance for point prediction, which is well studied. We benchmark three tabular foundation model variants against a diverse set of parametric, tree-based, and neural CDE baselines on 39 real-world datasets, across training sizes from 50 to 20,000, using six metrics covering density accuracy, calibration, and computation time. Across all sample sizes, foundation models achieve the best CDE loss, log-likelihood, and CRPS on the large majority of datasets tested. Calibration is competitive at small sample sizes but, for some metrics and datasets, lags behind task-specific neural baselines at larger sample sizes. However, we show that a post-hoc PIT recalibration step improves absolute calibration of the foundation models while leaving their accuracy advantage intact. In a photometric redshift case study using SDSS DR18, TabPFN exposed to 50,000 training galaxies outperforms all baselines trained on the full 500,000-galaxy dataset. Taken together, these results establish tabular foundation models as strong off-the-shelf conditional density estimators.
Related
- Conformal Prediction and Conditional Coverage for Tabular Foundation Models
- Unrestricted conditional maximum likelihood estimation for a Fréchet regression model
- Joint Conditional Maximum Likelihood Estimation of Beta Regression for Panel Data with Higher-Order Autoregressive Errors
- Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs
- Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Data
BibTeX
@article{izbicki2026benchmarking,
title = {{Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression}},
author = {Rafael Izbicki and Pedro Luiz Coelho Rodrigues},
journal = {Transactions on Machine Learning Research},
year = {2026},
month = sep,
url = {https://hal.science/hal-05755173v1},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression
AU - Rafael Izbicki
AU - Pedro Luiz Coelho Rodrigues
PY - 2026
DA - 2026/09/18
JO - Transactions on Machine Learning Research
UR - https://hal.science/hal-05755173v1
AB - Conditional density estimation (CDE) -recovering the full conditional distribution of a response given tabular covariates -is essential in settings with heteroscedasticity, multimodality, or asymmetric uncertainty. Recent tabular foundation models, such as TabPFN and TabICL, naturally produce predictive distributions, but their effectiveness as generalpurpose CDE methods has not been systematically evaluated -unlike their performance for point prediction, which is well studied. We benchmark three tabular foundation model variants against a diverse set of parametric, tree-based, and neural CDE baselines on 39 real-world datasets, across training sizes from 50 to 20,000, using six metrics covering density accuracy, calibration, and computation time. Across all sample sizes, foundation models achieve the best CDE loss, log-likelihood, and CRPS on the large majority of datasets tested. Calibration is competitive at small sample sizes but, for some metrics and datasets, lags behind task-specific neural baselines at larger sample sizes. However, we show that a post-hoc PIT recalibration step improves absolute calibration of the foundation models while leaving their accuracy advantage intact. In a photometric redshift case study using SDSS DR18, TabPFN exposed to 50,000 training galaxies outperforms all baselines trained on the full 500,000-galaxy dataset. Taken together, these results establish tabular foundation models as strong off-the-shelf conditional density estimators.
SN - 2835-8856
ER -