Transferable Graph Metanetworks
Abstract
A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of one or a few fixed sizes and evaluates them in-distribution. The few attempts at out-of-distribution size generalization remain limited in scope and have achieved only modest success. Consequently, the potential efficiency gains of training on small networks and evaluating on much larger ones remain largely unrealized. We propose Transferable Graph Metanetworks, which extend the graph metanetwork paradigm with a set of modifications that make performance transferable across input networks of different widths. The modifications follow two principles: invariance to the ways in which networks of different widths represent the same function, and continuity, such that weights representing similar functions receive similar predictions. We further study whether size generalization is possible for input networks trained independently from random initialization. Empirically, our modifications significantly improve size generalization on every task we consider. Performance is strongest on input networks trained under the maximal-update parameterization (μP), where it remains robust up to 42× the training width. Theoretically, we explain these observations with infinite-width limit theory: we prove size-generalization guarantees for our model on μP-trained inputs, and explain why it can fail under other parameterizations.
Related
- Efficient and Accurate Lift-and-Decoupling Generalized Transferable Neural Networks for Elliptic and Moving Interface Problems
- An Adaptive Transferable Neural Network for Partial Differential Equations with Anisotropic or Sharp Solutions
- A neural density functional for polymer electrolytes with exact structure and transferable Coulomb coupling
- PhysBridge: Response-supervised grid pretraining with physics-constrained local correction for transferable power-system intelligence
- Edge‐Coloring a Graph G so That Every Copy of a Graph H Has an Odd Color Class
BibTeX
@misc{ma2026transferable,
title = {{Transferable Graph Metanetworks}},
author = {Yuxin Ma and Adir Dayan and Yam Eitan and Haggai Maron and Soledad Villar},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.00420},
eprint = {2610.00420},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.00420},
}
RIS
TY - UNPB
TI - Transferable Graph Metanetworks
AU - Yuxin Ma
AU - Adir Dayan
AU - Yam Eitan
AU - Haggai Maron
AU - Soledad Villar
PY - 2026
DA - 2026/10/01
PB - arXiv
DO - 10.48550/arxiv.2610.00420
UR - https://arxiv.org/abs/2610.00420
AB - A weight space network (or metanetwork) takes the weights of another neural network as input and predicts properties of it. Most prior work trains such models on input networks of one or a few fixed sizes and evaluates them in-distribution. The few attempts at out-of-distribution size generalization remain limited in scope and have achieved only modest success. Consequently, the potential efficiency gains of training on small networks and evaluating on much larger ones remain largely unrealized. We propose Transferable Graph Metanetworks, which extend the graph metanetwork paradigm with a set of modifications that make performance transferable across input networks of different widths. The modifications follow two principles: invariance to the ways in which networks of different widths represent the same function, and continuity, such that weights representing similar functions receive similar predictions. We further study whether size generalization is possible for input networks trained independently from random initialization. Empirically, our modifications significantly improve size generalization on every task we consider. Performance is strongest on input networks trained under the maximal-update parameterization (μP), where it remains robust up to 42× the training width. Theoretically, we explain these observations with infinite-width limit theory: we prove size-generalization guarantees for our model on μP-trained inputs, and explain why it can fail under other parameterizations.
ER -