Skip to content
DraconiticResearch
Search Sources API
194,161 studies31 sourcesUpdated 3 min ago
PreprintCC BY 4.0arXiv

A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training

  • Andrea Angino
  • Bindi Çapriqi
  • Shega Likaj
  • Ken Trotti
  • Rolf Krause
Open at sourcePDFDOICite

Abstract

Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS.

Related

  1. An Inexact Halpern-accelerated Preconditioned Generalized Proximal Point Algorithm for the Maximal Monotone Inclusion Problem
    Lei Yang, Haihang Lan, Di Hou +2
    arXivPreprintarXiv licenceFull textOptimisation & controlarXiv
  2. Preconditioned Physics-Informed Neural Operator Training
    Shizheng Wen, Siddhartha Mishra, Marius Zeinhofer
    arXivPreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
  3. Preconditioned Latent Variable Proximal Point Method for Optimal Control Problems with Control Constraints
    Nico Haaf, Roland Herzog
    arXivPreprintCC BY-SA 4.0Full textNumerical analysis & scientific computingOptimisation & controlarXiv
  4. A Parallel Algebraic Multigrid Preconditioned Iterative Method Based on an MPI Process Grouping Strategy for Asynchronous Concurrent Execution of the Setup and Solve Phases
    Chunsheng Feng, Shuai Liu, Linjie Zhang +1
    arXivPreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
  5. A Preconditioned Semismooth Newton Method for Box-Constrained Elliptic Optimal Control Problems with Random Coefficients
    Zhendong Li, Akwum Onwunta, Hailing Wang
    arXivPreprintCC BY 4.0Full textAnalysis & PDEsNumerical analysis & scientific computingarXiv

Cite

BibTeXRIS
BibTeX
@misc{angino2026nonmonotone,
  title = {{A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training}},
  author = {Andrea Angino and Bindi Çapriqi and Shega Likaj and Ken Trotti and Rolf Krause},
  howpublished = {arXiv},
  year = {2026},
  month = may,
  doi = {10.48550/arxiv.2605.14860},
  eprint = {2605.14860},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.14860},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - A Non-Monotone Preconditioned Trust-Region Method for Neural Network Training
AU  - Andrea Angino
AU  - Bindi Çapriqi
AU  - Shega Likaj
AU  - Ken Trotti
AU  - Rolf Krause
PY  - 2026
DA  - 2026/05/14
PB  - arXiv
DO  - 10.48550/arxiv.2605.14860
UR  - https://arxiv.org/abs/2605.14860
AB  - Training deep neural networks at scale can benefit from domain decomposition, where the network is split into subdomains trained in parallel and coupled by a global trust-region mechanism. Building on the Additively Preconditioned Trust-Region Strategy (APTS), we propose a non-monotone variant with a nonlinear additive Schwarz preconditioner that combines parallel subdomain corrections with global coarse-space directions. A windowed acceptance criterion allows controlled objective increases, avoiding needless rejection of effective coarse steps. The resulting non-monotone APTS (NAPTS) preserves accuracy while reducing CPU time by 30\% and cutting rejected steps to one third of those in APTS.
ER  -