Distributed online learning with growing dimensions for stochastic large regression
Abstract
This paper investigates distributed online estimation and prediction for stochastic large regression models with infinitely many unknown parameters. Existing distributed estimation methods are mostly restricted to finite-dimensional parameter spaces, making them inadequate for emerging infinite-dimensional modeling applications. To tackle this problem, this paper proposes a fully distributed online recursive least squares algorithm with dynamically growing model dimension.The algorithm is constructed with an incrementally augmentable recursive structure, where new parameter dimensions can be embedded in the recursion via zero-padded covariance and estimate augmentation without re-initialization.The almost sure convergence of the algorithm is established under a weak cooperative non-persistent excitation condition, without requiring independence or stationarity assumptions on the regressors.This condition integrates spatio-temporal network information and extends existing finite-dimensional results. The paper further derives an almost sure upper bound on the accumulated prediction regret without any excitation conditions. It is proven that the average regret converges to zero almost surely for a broad class of dimension growth sequences, with the optimal finite-dimensional regret bound recovered as a special case. Finally, numerical simulations are conducted to verify the effectiveness of our theoretical findings.
Related
- Asymptotic Risk of Learning-Rate Schedules for Least-Squares Regression with Discretely Distributed Features
- Non-uniform exponential growth and the decay of growth rates in growing dimensions
- Assouad Dimensions and Spectra of Graphs of Weierstrass-Type Functions with Rapidly Growing Frequencies
- Clouded Representatives and Score Matching for Distributed Quantile Regression
- Distributed Selective Inference for Quantile Regression
BibTeX
@misc{gan2026distributed,
title = {{Distributed online learning with growing dimensions for stochastic large regression}},
author = {Die Gan and Zhixin Liu},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.39197},
eprint = {2610.39197},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.39197},
}
RIS
TY - UNPB
TI - Distributed online learning with growing dimensions for stochastic large regression
AU - Die Gan
AU - Zhixin Liu
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.39197
UR - https://arxiv.org/abs/2610.39197
AB - This paper investigates distributed online estimation and prediction for stochastic large regression models with infinitely many unknown parameters. Existing distributed estimation methods are mostly restricted to finite-dimensional parameter spaces, making them inadequate for emerging infinite-dimensional modeling applications. To tackle this problem, this paper proposes a fully distributed online recursive least squares algorithm with dynamically growing model dimension.The algorithm is constructed with an incrementally augmentable recursive structure, where new parameter dimensions can be embedded in the recursion via zero-padded covariance and estimate augmentation without re-initialization.The almost sure convergence of the algorithm is established under a weak cooperative non-persistent excitation condition, without requiring independence or stationarity assumptions on the regressors.This condition integrates spatio-temporal network information and extends existing finite-dimensional results. The paper further derives an almost sure upper bound on the accumulated prediction regret without any excitation conditions. It is proven that the average regret converges to zero almost surely for a broad class of dimension growth sequences, with the optimal finite-dimensional regret bound recovered as a special case. Finally, numerical simulations are conducted to verify the effectiveness of our theoretical findings.
ER -