Dynamical coarse-graining of neural-network training
Abstract
Dynamical coarse-graining of neural networks gives macroscopic observables and enables training to be predicted, and controlled in a state space far smaller than the parameter space. In this paper, we first define the learning operator following the training law of neural networks and establish a descent theorem for the learning operator, which serves to determine whether the operator is sufficient to characterize the learning dynamics. Then the exact closure theorem and the dynamic coarse-graining theorem are presented in sequence. The three theorems established in this paper provide a systematic framework for constructing and assessing dynamical coarse-grained models of neural-network training. Finally, we demonstrate the applicability of the theorems with a finite-width nonlinear ReLU network. Its dynamic coarse-grained model for a fixed pattern is exactly derived and composed of five independent moments. The model exactly predict the training outputs and evolving learning operator. The framework may also guide the development of approximate coarse-grained models for more general neural networks when exact closure is unavailable.
Related
- Conditional preservation of chimera states under equitable network coarse-graining
- Only Linear Constraints Survive Coarse-Graining: Evaluating Physics-Constrained Neural Operators on Stochastically Forced Turbulence
- Coarse-graining far-from-equilibrium dynamics using oblique projectors
- Coarse-graining to create minimalist models for dynamic, end-linked star-polymer networks
- Emergent Sparsity: How Many Effective Variables Survive Coarse-Graining in AlphaEarth Embeddings?
BibTeX
@misc{wang2026dynamical,
title = {{Dynamical coarse-graining of neural-network training}},
author = {Lin-Juan Wang and Da Li and Huiling Duan and Jianxiang Wang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36180},
eprint = {2610.36180},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36180},
}
RIS
TY - UNPB
TI - Dynamical coarse-graining of neural-network training
AU - Lin-Juan Wang
AU - Da Li
AU - Huiling Duan
AU - Jianxiang Wang
PY - 2026
DA - 2026/10/26
PB - arXiv
DO - 10.48550/arxiv.2610.36180
UR - https://arxiv.org/abs/2610.36180
AB - Dynamical coarse-graining of neural networks gives macroscopic observables and enables training to be predicted, and controlled in a state space far smaller than the parameter space. In this paper, we first define the learning operator following the training law of neural networks and establish a descent theorem for the learning operator, which serves to determine whether the operator is sufficient to characterize the learning dynamics. Then the exact closure theorem and the dynamic coarse-graining theorem are presented in sequence. The three theorems established in this paper provide a systematic framework for constructing and assessing dynamical coarse-grained models of neural-network training. Finally, we demonstrate the applicability of the theorems with a finite-width nonlinear ReLU network. Its dynamic coarse-grained model for a fixed pattern is exactly derived and composed of five independent moments. The model exactly predict the training outputs and evolving learning operator. The framework may also guide the development of approximate coarse-grained models for more general neural networks when exact closure is unavailable.
ER -