Variance-Reduced Stochastic Quasi-Newton SQCQP Method for Nonlinear Programming Problems
Abstract
In this paper, we propose a variance-reduced stochastic quasi-Newton sequential quadratic constrained quadratic programming method (VR-QN-SQCQP) for finite-sum nonlinear optimization with equality and inequality constraints. Exact nonlinear-constraint Hessians are replaced entirely by safeguarded symmetric rank-one (SR1) curvature models built from first-order constraint gradients, while PAGE estimates the objective gradient and an exact L1 merit function provides globalization. Under explicit regularity, boundedness, relative QCQP-accuracy, and step-length assumptions, the analysis establishes almost-sure subsequential convergence to Karush--Kuhn--Tucker (KKT) points and, under stronger uniform step-length and error-control conditions, an expected O(ε−2) outer-iteration bound for the squared KKT residual. A conditional local stability result also compares learned-curvature directions with their exact-Hessian counterparts. Numerical experiments compare VR-QN-SQCQP against Exact-SQCQP-PAGE on Sparse PCA using 253,680 CDC BRFSS 2015 observations and on a dense nonlinear benchmark. Over ten seeds, both methods converge in all Sparse PCA runs, while on the dense benchmark the proposed method converges in 10/10 runs versus 9/10 for the exact-Hessian variant. The proposed method performs zero exact constraint-Hessian evaluations and reduces median curvature-construction time on the dense benchmark from 0.1230 s to 0.0111 s. Overall runtime benefits remain problem-dependent.
Related
- Adaptive Variance-Reduced Stochastic Sequential (Quadratic Constrained) Quadratic Programming
- Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle
- Variance and volatility swap valuation with nonlinear liquidity-feedback stochastic volatility and regime switching
- VGAS: Variance-Reduced Guidance and Adaptive Selection for Training-Free Reward Alignment in Discrete Diffusion
- Variance-reduced First-order Methods for Finite-sum Function Constrained Convex Optimization
BibTeX
@misc{tiwari2026variancereduced,
title = {{Variance-Reduced Stochastic Quasi-Newton SQCQP Method for Nonlinear Programming Problems}},
author = {Gopal Tiwari and Md Abu Talhamainuddin Ansary},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.04403},
eprint = {2611.04403},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.04403},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Variance-Reduced Stochastic Quasi-Newton SQCQP Method for Nonlinear Programming Problems
AU - Gopal Tiwari
AU - Md Abu Talhamainuddin Ansary
PY - 2026
DA - 2026/11/03
PB - arXiv
DO - 10.48550/arxiv.2611.04403
UR - https://arxiv.org/abs/2611.04403
AB - In this paper, we propose a variance-reduced stochastic quasi-Newton sequential quadratic constrained quadratic programming method (VR-QN-SQCQP) for finite-sum nonlinear optimization with equality and inequality constraints. Exact nonlinear-constraint Hessians are replaced entirely by safeguarded symmetric rank-one (SR1) curvature models built from first-order constraint gradients, while PAGE estimates the objective gradient and an exact L_1 merit function provides globalization. Under explicit regularity, boundedness, relative QCQP-accuracy, and step-length assumptions, the analysis establishes almost-sure subsequential convergence to Karush--Kuhn--Tucker (KKT) points and, under stronger uniform step-length and error-control conditions, an expected O(ε^−2) outer-iteration bound for the squared KKT residual. A conditional local stability result also compares learned-curvature directions with their exact-Hessian counterparts. Numerical experiments compare VR-QN-SQCQP against Exact-SQCQP-PAGE on Sparse PCA using 253,680 CDC BRFSS 2015 observations and on a dense nonlinear benchmark. Over ten seeds, both methods converge in all Sparse PCA runs, while on the dense benchmark the proposed method converges in 10/10 runs versus 9/10 for the exact-Hessian variant. The proposed method performs zero exact constraint-Hessian evaluations and reduces median curvature-construction time on the dense benchmark from 0.1230 s to 0.0111 s. Overall runtime benefits remain problem-dependent.
ER -