Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs
Abstract
Artificial intelligence offers statisticians a new paradigm for modelling and analysing complex datasets. Agents based on large language models can reason about data and automate key stages of the statistical modelling workflow, from model fitting to validation. In this paper we propose a method that uses such an agent to estimate the conditional density of a response given a vector of covariates from an iid sample. The agent examines the sample, chooses a model for the conditional distribution of the response and the procedure to fit it, and returns the estimator as a short computer program. The program is built from a library of probability distributions. The returned law can therefore be a parametric distribution, a mixture or a multivariate factorization, with distribution parameters given by explicit formulas in the covariates. We call this approach agentic distributional regression (ADR). Unlike in the classical approach, the statistician no longer chooses the estimator, but asks an agent to build one. Treating the returned program as a random conditional density estimator, we derive a finite-sample error bound under assumptions on the number of admissible models and on the accuracy of the agent's fit, and an oracle inequality for the selection of a model among fitted candidates using independent validation data. Finally, assuming that, as the sample size grows, the agent selects the same smooth parametric model with probability tending to one and fits it accurately, we show that the estimated parameters are asymptotically normal. In simulation studies, on twelve real datasets and on a new dataset whose responses postdate the agents' training data, ADR with Claude Opus 5.5 has the best average performance among the existing methods from the statistical literature with which it is compared.
Related
- Orthogonal Inference for Conditional Z-Estimation without Cross-Fitting: A Distributional Nearest-Neighbor Approach
- Unrestricted conditional maximum likelihood estimation for a Fréchet regression model
- Joint Conditional Maximum Likelihood Estimation of Beta Regression for Panel Data with Higher-Order Autoregressive Errors
- Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression
- On the Calibration of Isotonic Distributional Regression
BibTeX
@misc{tedesco2026agentic,
title = {{Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs}},
author = {Lorenzo Tedesco and Francesco Finazzi},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.37611},
eprint = {2610.37611},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.37611},
}
RIS
TY - UNPB
TI - Agentic Distributional Regression: Conditional Density Estimation through Agent-Written Programs
AU - Lorenzo Tedesco
AU - Francesco Finazzi
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.37611
UR - https://arxiv.org/abs/2610.37611
AB - Artificial intelligence offers statisticians a new paradigm for modelling and analysing complex datasets. Agents based on large language models can reason about data and automate key stages of the statistical modelling workflow, from model fitting to validation. In this paper we propose a method that uses such an agent to estimate the conditional density of a response given a vector of covariates from an iid sample. The agent examines the sample, chooses a model for the conditional distribution of the response and the procedure to fit it, and returns the estimator as a short computer program. The program is built from a library of probability distributions. The returned law can therefore be a parametric distribution, a mixture or a multivariate factorization, with distribution parameters given by explicit formulas in the covariates. We call this approach agentic distributional regression (ADR). Unlike in the classical approach, the statistician no longer chooses the estimator, but asks an agent to build one. Treating the returned program as a random conditional density estimator, we derive a finite-sample error bound under assumptions on the number of admissible models and on the accuracy of the agent's fit, and an oracle inequality for the selection of a model among fitted candidates using independent validation data. Finally, assuming that, as the sample size grows, the agent selects the same smooth parametric model with probability tending to one and fits it accurately, we show that the estimated parameters are asymptotically normal. In simulation studies, on twelve real datasets and on a new dataset whose responses postdate the agents' training data, ADR with Claude Opus 5.5 has the best average performance among the existing methods from the statistical literature with which it is compared.
ER -