Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks
Abstract
Graph neural networks (GNNs) have emerged as a powerful framework for learning from graph-structured data. However, their efficient training remains challenging, particularly in distributed computing environments. This challenge arises from the use of message passing, which couples all graph nodes, leading to expensive optimization steps, high memory requirements, and substantial communication overhead. To alleviate these limitations, we propose a novel domain-decomposition (DD) variant of AG2m, an AdaGrad method enhanced with second-order curvature information and momentum, denoted by DD-AG2m. The proposed DD-AG2m alternates between AG2m optimization on the original (global) graph and AG2m optimization on the partitioned graphs. To incorporate global information at reduced cost, we further introduce a two-level variant (2DD-AG2m) that performs global optimization steps on a coarse graph obtained by randomly subsampling nodes within each subdomain. Numerical
Related
- Scalable partial information decomposition for symptom networks via supervised embeddings
- A Scalable Learned Iterative Solver for Nonlinear MPC via Temporal Decomposition
- The geometric convergence of a parallel domain decomposition method on manifolds
- Analysis of an inexact Schwarz domain decomposition method in 3D
- Energy-minimizing domain decomposition
BibTeX
@misc{varnas2026twolevel,
title = {{Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks}},
author = {Laurynas Varnas and Julien Herrmann and Alexander Heinlein and Serge Gratton and Alena Kopaničáková},
howpublished = {HAL},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.22575},
url = {https://hal.science/hal-05753812v1},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Two-level domain-decomposition AdaGrad method for scalable training of graph neural networks
AU - Laurynas Varnas
AU - Julien Herrmann
AU - Alexander Heinlein
AU - Serge Gratton
AU - Alena Kopaničáková
PY - 2026
DA - 2026/08/23
PB - HAL
DO - 10.48550/arxiv.2608.22575
UR - https://hal.science/hal-05753812v1
AB - Graph neural networks (GNNs) have emerged as a powerful framework for learning from graph-structured data. However, their efficient training remains challenging, particularly in distributed computing environments. This challenge arises from the use of message passing, which couples all graph nodes, leading to expensive optimization steps, high memory requirements, and substantial communication overhead. To alleviate these limitations, we propose a novel domain-decomposition (DD) variant of AG2m, an AdaGrad method enhanced with second-order curvature information and momentum, denoted by DD-AG2m. The proposed DD-AG2m alternates between AG2m optimization on the original (global) graph and AG2m optimization on the partitioned graphs. To incorporate global information at reduced cost, we further introduce a two-level variant (2DD-AG2m) that performs global optimization steps on a coarse graph obtained by randomly subsampling nodes within each subdomain. Numerical
ER -