An Adaptive Augmented Lagrangian Method for Deterministic and Stochastic Nonconvex Optimization
Abstract
We present an inexact Augmented Lagrangian algorithm for solving nonlinear, non-convex optimization problems. Unlike most recently proposed Augmented Lagrangian methods with worst-case complexity guarantees, we utilize adaptive penalty parameter updates and full dual stepsizes. We show that the method matches the best known worst-case complexity results for Augmented Lagrangian methods (up to logarithmic factors) when both the function and constraints are deterministic, when the function is stochastic and the constraints are deterministic, and when both are stochastic. Experiments on CUTEst test problems confirm the practical advantages of the proposed approach over Augmented Lagrangian methods with non-adaptive penalty parameters and/or short dual step sizes in the deterministic setting. Numerical results on stochastic constrained optimization problems in machine learning also confirm these findings.
Related
- Stochastic Augmented Lagrangian Framework with Second-Order Convergence Guarantees for Nonconvex Expectation-Constrained Optimization
- Nonconvex Composite Functional Constraints via First-Order Augmented Lagrangian Methods under Local Regularity
- ALDER: A Single-Loop Augmented-Lagrangian Damped Extragradient Method for Nonconvex--Concave Minimax Problems with Nonlinear Coupled Constraints
- Optimal Deterministic First-Order Oracle Complexity for Nonconvex-Concave Minimax Optimization
- An Inexact Augmented Lagrangian Method for (L0, L1)-Smooth Convex Optimization
BibTeX
@misc{liu2026adaptive,
title = {{An Adaptive Augmented Lagrangian Method for Deterministic and Stochastic Nonconvex Optimization}},
author = {Tianzhu Liu and Michael J. O'Neill},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.27676},
eprint = {2608.27676},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.27676},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - An Adaptive Augmented Lagrangian Method for Deterministic and Stochastic Nonconvex Optimization
AU - Tianzhu Liu
AU - Michael J. O'Neill
PY - 2026
DA - 2026/08/27
PB - arXiv
DO - 10.48550/arxiv.2608.27676
UR - https://arxiv.org/abs/2608.27676
AB - We present an inexact Augmented Lagrangian algorithm for solving nonlinear, non-convex optimization problems. Unlike most recently proposed Augmented Lagrangian methods with worst-case complexity guarantees, we utilize adaptive penalty parameter updates and full dual stepsizes. We show that the method matches the best known worst-case complexity results for Augmented Lagrangian methods (up to logarithmic factors) when both the function and constraints are deterministic, when the function is stochastic and the constraints are deterministic, and when both are stochastic. Experiments on CUTEst test problems confirm the practical advantages of the proposed approach over Augmented Lagrangian methods with non-adaptive penalty parameters and/or short dual step sizes in the deterministic setting. Numerical results on stochastic constrained optimization problems in machine learning also confirm these findings.
ER -