Operator Splitting Methods with Online Scaling
Abstract
This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting, and the alternating direction method of multipliers. We interpret these splitting methods as preconditioned gradient descent applied to their corresponding envelope functions and use online learning to adaptively learn the preconditioner during optimization. Our framework achieves asymptotically faster convergence rates and accelerates the solution of important optimization problems, including Lasso, ℓ1-regularized logistic regression, and quadratic programming. Extensive numerical experiments validate our theoretical results and demonstrate the practical effectiveness of the proposed approach.
Related
- Hopf Algebra of Graphs for Operator Splitting Methods
- Operator splitting methods for non-autonomous evolution equations comprising three parts, with application to the Bloch-Torrey equation of MRI
- Stochastic Gradient Methods with Online Scaling
- Finite-sample guarantees for data-driven operator splitting methods via martingale inequalities
- On the Local Linear Convergence of Operator Splitting Methods for Conic Programming
BibTeX
@misc{zhang2026operator,
title = {{Operator Splitting Methods with Online Scaling}},
author = {Wanyu Zhang and Wenzhi Gao and Madeleine Udell},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.10732},
eprint = {2609.10732},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.10732},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Operator Splitting Methods with Online Scaling
AU - Wanyu Zhang
AU - Wenzhi Gao
AU - Madeleine Udell
PY - 2026
DA - 2026/09/09
PB - arXiv
DO - 10.48550/arxiv.2609.10732
UR - https://arxiv.org/abs/2609.10732
AB - This paper develops a principled framework for automatically tuning preconditioners in operator splitting methods, including forward-backward splitting, Douglas-Rachford splitting, and the alternating direction method of multipliers. We interpret these splitting methods as preconditioned gradient descent applied to their corresponding envelope functions and use online learning to adaptively learn the preconditioner during optimization. Our framework achieves asymptotically faster convergence rates and accelerates the solution of important optimization problems, including Lasso, ℓ_1-regularized logistic regression, and quadratic programming. Extensive numerical experiments validate our theoretical results and demonstrate the practical effectiveness of the proposed approach.
ER -