The Continuum Limit of the Deep Linear Network
Abstract
The Deep Linear Network (DLN) is a simple version of a Multilayer Perceptron (MLP) in which the activation function is the identity and the bias vector is the zero vector. We examine the behavior of the model when the number of weight matrices, which is finite in the original model, goes to infinity. We show that a plausible continuum limit is a dimensionally reduced version of the Yang-Mills equation on the torus, revealing an unexpected geometric interpretation of infinite-depth DLNs. While we do not prove this rigorously, it is a very pretty calculation. It also gives some insight into several features of the construction. We note that the choices of boundary conditions on the ends of the discrete series survive better in the continuum limit. Also, the regularizing functional which uses the moment map survives nicely in the continuum limit, whereas the regularizing functional using the size of approximating terms has no sensible limit. This surely is useful information for the choices we make in the finite-dimensional theory, and might also be useful in the more complicated MLP setting.
Related
- Structural Interpretability: Unifying Deep Network Analysis via Piecewise Linear Path Algebras
- The Continuum Limit Analysis of Causal Fermion Systems for Curved Spacetimes
- Continuum Limit for One-Dimensional Moiré Relaxation
- Inverse eigenvalue problem for discrete three-diagonal Schroedinger operator and the continuum limit in Tikhonov regularization approach
- Continuum Limit of Nonlocal Electrostatics in Random Media
BibTeX
@misc{pfaff2026continuum,
title = {{The Continuum Limit of the Deep Linear Network}},
author = {Catherine Eva Pfaff and Karen Uhlenbeck},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.00503},
eprint = {2611.00503},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.00503},
}
RIS
TY - UNPB
TI - The Continuum Limit of the Deep Linear Network
AU - Catherine Eva Pfaff
AU - Karen Uhlenbeck
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.00503
UR - https://arxiv.org/abs/2611.00503
AB - The Deep Linear Network (DLN) is a simple version of a Multilayer Perceptron (MLP) in which the activation function is the identity and the bias vector is the zero vector. We examine the behavior of the model when the number of weight matrices, which is finite in the original model, goes to infinity. We show that a plausible continuum limit is a dimensionally reduced version of the Yang-Mills equation on the torus, revealing an unexpected geometric interpretation of infinite-depth DLNs. While we do not prove this rigorously, it is a very pretty calculation. It also gives some insight into several features of the construction. We note that the choices of boundary conditions on the ends of the discrete series survive better in the continuum limit. Also, the regularizing functional which uses the moment map survives nicely in the continuum limit, whereas the regularizing functional using the size of approximating terms has no sensible limit. This surely is useful information for the choices we make in the finite-dimensional theory, and might also be useful in the more complicated MLP setting.
ER -