Structured Spectral Step-Sizes and Rebound-Aware Ordering for Gradient Methods
Abstract
The performance of gradient methods depends critically on step-size selection. Spectral step-size design involves constructing candidates from finite history and ordering them during iteration. We address both through a finite-moment spectral framework. For candidate construction, we relate Huang--Dai--Liu determinant pencils, limited-memory steepest-descent Krylov--Ritz extraction, pseudo-memory moment realization, and Gu--Du recovery of spectral nodes and component weights. The framework distinguishes generalized-eigenvalue recoverability, finite-window implementability, and component-weight interpretability. Under explicit positivity, rank-one, regularity, and exact-recovery assumptions, these properties become compatible in a common finite-moment regime, with the long, short, and mean LMSD constructions as representatives. For ordering, we identify spectral rebound: a step targeting a lower spectral scale may reactivate previously reduced high-frequency components. Idealized one-target, block, and recomputed memory-m models yield phase-count recurrences motivating a high-frequency-first strategy with spectral refreshes. We propose a rebound-aware spectral gradient method (RASG) that ranks Ritz candidates by frequency and estimated strength, and sets phase duration via a Ritz-uncertainty estimator. Two variants, RASG-KT and RASG-LMSD, use a Kato--Temple-inspired estimator and a long/short LMSD discrepancy. For strictly convex quadratics, global R-linear convergence holds under an admissibility safeguard. Experiments on ill-conditioned quadratic problems and two large-scale convex problems illustrate the behavior.
Related
- EQUIVARIANCE VERSUS FIRM NON-EXPANSIVENESS IN INCREMENTAL PROXIMAL GRADIENT METHODS FOR PLUG-AND-PLAY DENOISING
- On The Linear Convergence of Bregman Proximal Gradient Methods with Applications to Kullback--Leibler regression
- Average Energy Dissipation Rates of Implicit-Explicit Runge-Kutta Methods for Gradient Flow Problems
- Gradient estimates for the Green kernel under spectral Ricci bounds, and the stable Bernstein theorem in ℝ⁴
- A Strain-Structured SIS Model with Mutation: Spectral Thresholds and Long-Time Dynamics
BibTeX
@misc{zhou2026structured,
title = {{Structured Spectral Step-Sizes and Rebound-Aware Ordering for Gradient Methods}},
author = {Yijia Zhou and Ran Gu and Ya-xiang Yuan},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.25311},
eprint = {2606.25311},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.25311},
}
RIS
TY - UNPB
TI - Structured Spectral Step-Sizes and Rebound-Aware Ordering for Gradient Methods
AU - Yijia Zhou
AU - Ran Gu
AU - Ya-xiang Yuan
PY - 2026
DA - 2026/09/23
PB - arXiv
DO - 10.48550/arxiv.2606.25311
UR - https://arxiv.org/abs/2606.25311
AB - The performance of gradient methods depends critically on step-size selection. Spectral step-size design involves constructing candidates from finite history and ordering them during iteration. We address both through a finite-moment spectral framework. For candidate construction, we relate Huang--Dai--Liu determinant pencils, limited-memory steepest-descent Krylov--Ritz extraction, pseudo-memory moment realization, and Gu--Du recovery of spectral nodes and component weights. The framework distinguishes generalized-eigenvalue recoverability, finite-window implementability, and component-weight interpretability. Under explicit positivity, rank-one, regularity, and exact-recovery assumptions, these properties become compatible in a common finite-moment regime, with the long, short, and mean LMSD constructions as representatives. For ordering, we identify spectral rebound: a step targeting a lower spectral scale may reactivate previously reduced high-frequency components. Idealized one-target, block, and recomputed memory-m models yield phase-count recurrences motivating a high-frequency-first strategy with spectral refreshes. We propose a rebound-aware spectral gradient method (RASG) that ranks Ritz candidates by frequency and estimated strength, and sets phase duration via a Ritz-uncertainty estimator. Two variants, RASG-KT and RASG-LMSD, use a Kato--Temple-inspired estimator and a long/short LMSD discrepancy. For strictly convex quadratics, global R-linear convergence holds under an admissibility safeguard. Experiments on ill-conditioned quadratic problems and two large-scale convex problems illustrate the behavior.
ER -