Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle
Abstract
Stochastic variance reduced gradient (SVRG) is a variant of stochastic gradient descent and is a promising iterative method for solving large-scale inverse problems. Nevertheless, the development of theoretically grounded a posteriori stopping rules for SVRG remains an open challenge. In this work, we provide a convergence analysis of SVRG equipped with the discrepancy principle, the most well-known a posteriori stopping rule, for solving a class of linear inverse problems in Hilbert spaces. We establish the regularizing property of SVRG, and moreover, under suitable source conditions, we derive convergence rates of SVRG iterates. To the best of our knowledge, these are the first convergence rate results of any stochastic iterative method for inverse problems under the a posteriori stopping rule. The theoretical findings are supported by numerical experiments.
Related
- Minimax-Optimal Early Stopping for Continuous-Time SGD via the Discrepancy Principle
- A Smoothed Discrepancy Principle for Random Feature Methods and Neural Networks
- When is early stopping beneficial under label noise? A phase diagram for kernel gradient flow
- From smooth to discontinuous kernels: a variance transfer principle for hyperuniform processes
- A Jump-Modified Wasserstein Gradient Flow for Maximum Mean Discrepancy with Energy Kernels
BibTeX
@misc{jin2026early,
title = {{Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle}},
author = {Bangti Jin and Zehui Zhou},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.12396},
eprint = {2609.12396},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.12396},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle
AU - Bangti Jin
AU - Zehui Zhou
PY - 2026
DA - 2026/09/11
PB - arXiv
DO - 10.48550/arxiv.2609.12396
UR - https://arxiv.org/abs/2609.12396
AB - Stochastic variance reduced gradient (SVRG) is a variant of stochastic gradient descent and is a promising iterative method for solving large-scale inverse problems. Nevertheless, the development of theoretically grounded a posteriori stopping rules for SVRG remains an open challenge. In this work, we provide a convergence analysis of SVRG equipped with the discrepancy principle, the most well-known a posteriori stopping rule, for solving a class of linear inverse problems in Hilbert spaces. We establish the regularizing property of SVRG, and moreover, under suitable source conditions, we derive convergence rates of SVRG iterates. To the best of our knowledge, these are the first convergence rate results of any stochastic iterative method for inverse problems under the a posteriori stopping rule. The theoretical findings are supported by numerical experiments.
ER -