Scalable and Adaptive Distribution Regression via Random Features
Abstract
Distribution regression seeks to learn mappings from probability distributions to response variables and has emerged as a unifying framework for kernel methods, multi-instance learning, and set-valued learning. Despite its success, kernel-based distribution regression remains computationally expensive and sensitive to kernel choice, motivating scalable and adaptive representations. Building on random feature approximations, we develop a finite-sample analysis of Random Feature Distribution Regression (RaDR) under two-stage sampling. Our perturbation analysis separates the effects of distribution-level sampling, finite within-bag sampling, and random feature approximation. We establish sufficient feature-budget conditions under which RaDR preserves the order of our excess-risk upper bound for exact two-stage kernel distribution regression while maintaining computational efficiency. We further propose Adaptive Random Feature Distribution Regression (A-RaDR), a unified score-guided framework that adaptively allocates random features according to downstream learning objectives. Experiments on synthetic benchmarks and 41 real-world datasets demonstrate that RaDR delivers substantial computational savings while maintaining competitive predictive performance, whereas A-RaDR further improves learning accuracy in distribution regression while retaining the computational speed up. Together, these results establish a practical and theoretically grounded framework for adaptive, scalable distribution learning
Related
- Scalable Nonparametric Change-Plane Regression with Random Features: Estimation and Inference
- A Shape-Adaptive Gamma-Mixture Power Shanker Distribution: Boundary-Aware Inference, Censored Regression, and Predictive Validation
- Recursive moment updates for scalable Bayesian nonparametric regression
- Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Rates
- Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
BibTeX
@misc{wang2026scalable,
title = {{Scalable and Adaptive Distribution Regression via Random Features}},
author = {Yinsong Wang and Xiao Liu and Yu Ding},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.24892},
eprint = {2610.24892},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.24892},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Scalable and Adaptive Distribution Regression via Random Features
AU - Yinsong Wang
AU - Xiao Liu
AU - Yu Ding
PY - 2026
DA - 2026/10/17
PB - arXiv
DO - 10.48550/arxiv.2610.24892
UR - https://arxiv.org/abs/2610.24892
AB - Distribution regression seeks to learn mappings from probability distributions to response variables and has emerged as a unifying framework for kernel methods, multi-instance learning, and set-valued learning. Despite its success, kernel-based distribution regression remains computationally expensive and sensitive to kernel choice, motivating scalable and adaptive representations. Building on random feature approximations, we develop a finite-sample analysis of Random Feature Distribution Regression (RaDR) under two-stage sampling. Our perturbation analysis separates the effects of distribution-level sampling, finite within-bag sampling, and random feature approximation. We establish sufficient feature-budget conditions under which RaDR preserves the order of our excess-risk upper bound for exact two-stage kernel distribution regression while maintaining computational efficiency. We further propose Adaptive Random Feature Distribution Regression (A-RaDR), a unified score-guided framework that adaptively allocates random features according to downstream learning objectives. Experiments on synthetic benchmarks and 41 real-world datasets demonstrate that RaDR delivers substantial computational savings while maintaining competitive predictive performance, whereas A-RaDR further improves learning accuracy in distribution regression while retaining the computational speed up. Together, these results establish a practical and theoretically grounded framework for adaptive, scalable distribution learning
ER -