The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group

Abstract

The differential neural tangent kernel (DNTK) is a tool for analyzing the training dynamics of overparameterized physics informed neural networks. In this work, we investigate the symmetry of the infinite DNTK under an orthogonal group, for two layer and deep neural networks separately, based on the explicit expression of the DNTK and recursive formula, respectively. The results are for all activations with or without a bias term, and shed insight into the structure of the infinite width DNTK. Then we prove that the integral operators over the unit sphere induced by operators from the same group orbit share the spectrum, and their reproducing kernel Hilbert spaces differ only by an orthogonal transformation, and that the operator fixed DNTK exhibits zonal symmetry on the unit sphere for symmetric operators. Thus the eigenvalue decay rate of the associated integral operator can be derived for shallow networks with RePU activation and zero bias initialization.

A dual approach to L^p estimates for orthogonal martingales satisfying the differential subordination
Rodrigo Bañuelos, Adam Osękowski
arXiv1 Nov 2026PreprintarXiv licenceFull textAnalysis & PDEs ProbabilityarXiv
Counterexamples and symmetry for uneven orthogonal mass partitions in the plane
Leonardo Martínez-Sandoval
arXiv15 Oct 2026PreprintarXiv licenceFull textCombinatorics & discrete mathematics Geometry & topologyarXiv
On higher-order differential equations for Laguerre-Hahn orthogonal polynomials
Mohamed Khalfallah, Zélia da Rocha
arXiv19 Sep 2026PreprintarXiv licenceFull textAnalysis & PDEsarXiv
Learning Orthogonal Multi-Index Models Beyond Small Initialization: Incremental Learning, Competitive Dynamics and Symmetry
Mo Zhou, Weihang Xu, Simon S. Du +1
arXiv9 Sep 2026PreprintCC BY 4.0Full textStatisticsarXiv
Signature Kernel and Schwinger-Dyson Kernel Equations as Two-Parameter Rough Differential Equations
Thomas Cass, Dan Crisan, Andrea Iannucci +1
arXiv11 Oct 2026PreprintCC BY 4.0Full textNumerical analysis & scientific computing ProbabilityarXiv

Cite

BibTeX RIS

BibTeX

@misc{jin2026symmetry,
  title = {{The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group}},
  author = {Bangti Jin and Longjun Wu},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.34023},
  eprint = {2610.34023},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.34023},
}

RIS

TY  - UNPB
TI  - The Symmetry of Differential Neural Tangent Kernel under the Orthogonal Group
AU  - Bangti Jin
AU  - Longjun Wu
PY  - 2026
DA  - 2026/10/25
PB  - arXiv
DO  - 10.48550/arxiv.2610.34023
UR  - https://arxiv.org/abs/2610.34023
AB  - The differential neural tangent kernel (DNTK) is a tool for analyzing the training dynamics of overparameterized physics informed neural networks. In this work, we investigate the symmetry of the infinite DNTK under an orthogonal group, for two layer and deep neural networks separately, based on the explicit expression of the DNTK and recursive formula, respectively. The results are for all activations with or without a bias term, and shed insight into the structure of the infinite width DNTK. Then we prove that the integral operators over the unit sphere induced by operators from the same group orbit share the spectrum, and their reproducing kernel Hilbert spaces differ only by an orthogonal transformation, and that the operator fixed DNTK exhibits zonal symmetry on the unit sphere for symmetric operators. Thus the eigenvalue decay rate of the associated integral operator can be derived for shallow networks with RePU activation and zero bias initialization.
ER  -

Abstract

Related

Cite