Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting
Abstract
Nonparametric estimation of joint probability distributions is a core problem in statistics, which is notoriously challenging with high-dimensional data. Such challenges also arise with conditional distribution estimation and have led to flexible algorithms that aim to tackle the "curse of dimensionality" by exploiting predictive structures of lower dimension than the ambient space. In this paper, we study the use of such conditional distribution estimators to estimate joint distributions by factorizing the latter in an "autoregressive" way. We show that for loss functions satisfying a subadditivity property, bounds on the convergence rate of an autoregressive estimator can be derived from bounds on the convergence rate of the conditional distribution estimators. These bounds relate the difficulty of estimating the joint distribution to that of estimating the most challenging factor in the autoregressive factorization. When the conditional estimators adapt to sparsity in their conditioning sets, we characterize this difficulty through variable elimination in an undirected dependence graph. This allows bounding the rates of convergence by quantities related to the sparsity of the graph, and provides an additional motivation for the autoregressive approach. We then evaluate the practical performance of an autoregressive estimator based on parallel gradient boosting, a flexible nonparametric method for estimating conditional distributions. The estimator demonstrates competitive estimation performance on various simulated and real datasets, even without optimizing the factorization order. Together, these results support nonparametric autoregressive estimation as a practical approach to flexible joint distribution estimation.
Related
- Parallel gradient boosting for flexible estimation of conditional distributions
- Nonparametric Change-Point Detection and Inference for High-Dimensional Distributions
- Nonparametric hazard rate estimation with associated kernels and minimax bandwidth choice
- The importance of the change-of-variables integration technique in deriving the PDFs of important probability distributions
- Estimation and Inference for the Joint Autoregressive Quantile-Expected Shortfall Model
BibTeX
@misc{chapelle2026nonparametric,
title = {{Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting}},
author = {Rémy Chapelle},
howpublished = {HAL},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.31524},
eprint = {2610.31524},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05835906v1},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Nonparametric autoregressive estimation of probability distributions with parallel gradient boosting
AU - Rémy Chapelle
PY - 2026
DA - 2026/10/22
PB - HAL
DO - 10.48550/arxiv.2610.31524
UR - https://hal.science/hal-05835906v1
AB - Nonparametric estimation of joint probability distributions is a core problem in statistics, which is notoriously challenging with high-dimensional data. Such challenges also arise with conditional distribution estimation and have led to flexible algorithms that aim to tackle the "curse of dimensionality" by exploiting predictive structures of lower dimension than the ambient space. In this paper, we study the use of such conditional distribution estimators to estimate joint distributions by factorizing the latter in an "autoregressive" way. We show that for loss functions satisfying a subadditivity property, bounds on the convergence rate of an autoregressive estimator can be derived from bounds on the convergence rate of the conditional distribution estimators. These bounds relate the difficulty of estimating the joint distribution to that of estimating the most challenging factor in the autoregressive factorization. When the conditional estimators adapt to sparsity in their conditioning sets, we characterize this difficulty through variable elimination in an undirected dependence graph. This allows bounding the rates of convergence by quantities related to the sparsity of the graph, and provides an additional motivation for the autoregressive approach. We then evaluate the practical performance of an autoregressive estimator based on parallel gradient boosting, a flexible nonparametric method for estimating conditional distributions. The estimator demonstrates competitive estimation performance on various simulated and real datasets, even without optimizing the factorization order. Together, these results support nonparametric autoregressive estimation as a practical approach to flexible joint distribution estimation.
ER -