Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting
Abstract
Based on second-order inertial dynamics, we develop two accelerated Lagrange multiplier (LM)-based optimization methods: Linear LM-based second-order flow method and linear LM-based variable and operator splitting method for unconstrained non-convex optimization problems. Relaxed and adaptive variants of both methods are proposed to prevent degeneration of the Lagrange multiplier and to enable automatic adjustment of the time step size. Within the novel second-order LM-based optimization framework, the associated pseudo-energy functional is shown to dissipate monotonically along the iterative trajectory, and convergence of the generated iterates to stationary points is rigorously established. Extensive numerical experiments on function optimization and partial differential equation benchmark problems, including applications as optimizers for physics-informed neural networks (PINNs) and deep operator networks (DeepONets), demonstrate that the proposed methods effectively escape local minima, achieve high accuracy, and exhibit strong robustness with respect to the choice of the initial learning rate during neural network training.
Related
- Structure-preserving upwind Lagrange multiplier schemes for solid-state dewetting with a logarithmic Flory--Huggins potential
- GPU-Accelerated Bregman Douglas-Rachford Splitting for Discrete Optimal Transport
- Accelerated Convergence of a Second-Order Dynamical System and its Application to Splitting Algorithms for Comonotone Inclusions
- Accelerated Plug-and-Play Davis-Yin Splitting for Nonconvex Image Reconstruction
- Variable Metric Three Operator Splitting
BibTeX
@misc{li2026several,
title = {{Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting}},
author = {Xiaoli Li and Jianghua Liu and Zhiping Mao and Jingrong Wei and Xuan Zhao},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.39695},
eprint = {2609.39695},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.39695},
}
RIS
TY - UNPB
TI - Several Accelerated and Stable Pseudo-Energy-Dissipative Lagrange Multiplier Methods based on Second-order Flow and Variable Splitting
AU - Xiaoli Li
AU - Jianghua Liu
AU - Zhiping Mao
AU - Jingrong Wei
AU - Xuan Zhao
PY - 2026
DA - 2026/09/30
PB - arXiv
DO - 10.48550/arxiv.2609.39695
UR - https://arxiv.org/abs/2609.39695
AB - Based on second-order inertial dynamics, we develop two accelerated Lagrange multiplier (LM)-based optimization methods: Linear LM-based second-order flow method and linear LM-based variable and operator splitting method for unconstrained non-convex optimization problems. Relaxed and adaptive variants of both methods are proposed to prevent degeneration of the Lagrange multiplier and to enable automatic adjustment of the time step size. Within the novel second-order LM-based optimization framework, the associated pseudo-energy functional is shown to dissipate monotonically along the iterative trajectory, and convergence of the generated iterates to stationary points is rigorously established. Extensive numerical experiments on function optimization and partial differential equation benchmark problems, including applications as optimizers for physics-informed neural networks (PINNs) and deep operator networks (DeepONets), demonstrate that the proposed methods effectively escape local minima, achieve high accuracy, and exhibit strong robustness with respect to the choice of the initial learning rate during neural network training.
ER -