On Parameter Symmetries and Conservation Laws in Gradient Flow

@misc{nguyen2026parameter,
  title = {{On Parameter Symmetries and Conservation Laws in Gradient Flow}},
  author = {Khang Nguyen and Guido Montúfar},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.34549},
  eprint = {2609.34549},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.34549},
}

TY  - UNPB
TI  - On Parameter Symmetries and Conservation Laws in Gradient Flow
AU  - Khang Nguyen
AU  - Guido Montúfar
PY  - 2026
DA  - 2026/09/28
PB  - arXiv
DO  - 10.48550/arxiv.2609.34549
UR  - https://arxiv.org/abs/2609.34549
AB  - Parameter space symmetries and conservation laws play an important role in understanding the loss landscapes and implicit biases of neural networks. Inspired by Noether's theorem in physics, prior works have sought to derive conservation laws under gradient flow from parameter symmetries, but the scope and limitations of this connection remain unclear. We develop a unified geometric framework that clarifies the precise relationship between the two notions, including the conditions under which symmetries correspond to conservation laws. We introduce a notion of compositional identifiability and use it to establish a general inheritance principle for complete characterizations of symmetries and conservation laws in multilayer networks. We apply the framework to multi-head and grouped-query attention, polynomial neural networks, and square deep linear networks.
ER  -