A Riemannian Inertial Adaptive Gradient Method with Momentum Restart

Abstract

We propose RIAG-R, a Riemannian Inertial Adaptive Gradient method with gradient-triggered Restart, for optimization on Riemannian manifolds. RIAG-R combines three ingredients: an AdaGrad-type adaptive step-size that requires no knowledge of the Lipschitz constant; inertial extrapolation in the tangent space via the exponential map to accelerate convergence; and a restart rule, costing only one extra inner product per iteration, that detects and corrects the momentum-saturation pathology induced by manifold curvature. We establish a per-iteration descent inequality, an optimal O(ε⁻²)-iteration complexity, a convergence result for cluster points of the generated sequence, and a linear convergence rate under the Riemannian Polyak-Łojasiewicz inequality. Numerical experiments on four representative matrix manifold optimization problems including sphere, Stiefel, symmetric positive-definite, and Grassmann manifolds demonstrate that RIAG-R consistently outperforms non-inertial and restart-free inertial baselines, confirming that adaptive restart is essential to realizing the benefits of momentum on curved spaces.

Restart and Adaptive Acceleration in Stochastic Gradient Methods
Ali Elhishi, Chistophe Roux, Alexandre d'Aspremont
HAL19 Jun 2026PreprintarXiv licenceFull textOptimisation & controlHALarXiv
On the Nonoccurrence of Gradient Restart in Nesterov Accelerated Gradient Method
Chenglong Bao, Liang Chen, Jiahong Li +1
arXiv19 Oct 2026PreprintarXiv licenceFull textOptimisation & controlarXiv
Adaptive Restart under Unknown KL Geometry in Stochastic Weakly-Convex Optimization
Binh T. Nguyen, Khoa D. Nguyen, Bao Thai Huynh
arXiv22 Oct 2026PreprintCC BY-NC-ND 4.0Full textOptimisation & controlarXiv
Sub-Laplacian Comparison Theorem and Gradient Estimates on Complete Noncompact Contact Riemannian Manifolds
Guilin Yang, Biqiang Zhao, Weihao Zheng
arXiv22 Oct 2026PreprintarXiv licenceFull textGeometry & topologyarXiv
Saturating spaces for the squared-gradient nonlinearity on compact Riemannian manifolds via Nash embeddings
Rémi Buffe, Alessandro Duca, Hugo Parada
HAL21 Oct 2026PreprintCC BY 4.0Full textAnalysis & PDEs Optimisation & controlHAL

Cite

BibTeX RIS

BibTeX

@misc{jolaoso2026riemannian,
  title = {{A Riemannian Inertial Adaptive Gradient Method with Momentum Restart}},
  author = {Lateef O. Jolaoso and Professer V. Ndlovu and Maggie Aphane},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.32997},
  eprint = {2609.32997},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.32997},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - A Riemannian Inertial Adaptive Gradient Method with Momentum Restart
AU  - Lateef O. Jolaoso
AU  - Professer V. Ndlovu
AU  - Maggie Aphane
PY  - 2026
DA  - 2026/09/26
PB  - arXiv
DO  - 10.48550/arxiv.2609.32997
UR  - https://arxiv.org/abs/2609.32997
AB  - We propose RIAG-R, a Riemannian Inertial Adaptive Gradient method with gradient-triggered Restart, for optimization on Riemannian manifolds. RIAG-R combines three ingredients: an AdaGrad-type adaptive step-size that requires no knowledge of the Lipschitz constant; inertial extrapolation in the tangent space via the exponential map to accelerate convergence; and a restart rule, costing only one extra inner product per iteration, that detects and corrects the momentum-saturation pathology induced by manifold curvature. We establish a per-iteration descent inequality, an optimal O(ε^−2)-iteration complexity, a convergence result for cluster points of the generated sequence, and a linear convergence rate under the Riemannian Polyak-Łojasiewicz inequality. Numerical experiments on four representative matrix manifold optimization problems including sphere, Stiefel, symmetric positive-definite, and Grassmann manifolds demonstrate that RIAG-R consistently outperforms non-inertial and restart-free inertial baselines, confirming that adaptive restart is essential to realizing the benefits of momentum on curved spaces.
ER  -

Abstract

Related

Cite