Failure of entropic selection for diagonal matrix completion in deep linear networks
Abstract
We study the free-energy gradient flow associated with the depth-N deep linear network on the space of invertible real d× d matrices. The loss observes only the diagonal, Ed(W)=1/2∑i=1d(Wii−1)2, and the regularizer is the Boltzmann entropy of the balanced factorization fiber computed by Menon and Yu. This is a natural higher-dimensional version of a matrix-completion problem posed by Menon as a test of whether entropy selects among a noncompact family of minimizers. For diagonal completion, every width d≥2, depth N>2, and inverse temperature β>0, we prove that the free energy is unbounded below and has exactly 2d full-rank critical points, one in each diagonal sign chamber. Every critical point is diagonal and hyperbolic. Its unstable dimension is binom d2 and its stable dimension is d(d+1)/2. We also prove that a full-rank trajectory cannot converge to a finite rank-deficient matrix. Consequently, almost every full-rank initial condition has an unbounded forward orbit. Thus finite-temperature fiber entropy does not provide an equilibrium selection principle for diagonal completion.
Related
- Minimum-makespan completion and vertex selection leave the Wang-Sitters constant at 11/6
- Near-diagonal asymptotics and the failure of strong universality in random Čech persistence
- Implicit Bias of (Stochastic) Gradient Descent on Deep Linear Diagonal Networks
- Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks
- Relevance Propagation as Entropic Optimal Transport on Filippov Neural Networks: Sinkhorn Propagation and Region Relevance in Image Classifiers
BibTeX
@misc{trevino2026failure,
title = {{Failure of entropic selection for diagonal matrix completion in deep linear networks}},
author = {Rodrigo Treviño},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.05551},
eprint = {2610.05551},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05551},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Failure of entropic selection for diagonal matrix completion in deep linear networks
AU - Rodrigo Treviño
PY - 2026
DA - 2026/10/04
PB - arXiv
DO - 10.48550/arxiv.2610.05551
UR - https://arxiv.org/abs/2610.05551
AB - We study the free-energy gradient flow associated with the depth-N deep linear network on the space of invertible real d× d matrices. The loss observes only the diagonal, E_d(W)=1/2∑_i=1^d(W_ii−1)^2, and the regularizer is the Boltzmann entropy of the balanced factorization fiber computed by Menon and Yu. This is a natural higher-dimensional version of a matrix-completion problem posed by Menon as a test of whether entropy selects among a noncompact family of minimizers. For diagonal completion, every width d≥2, depth N>2, and inverse temperature β>0, we prove that the free energy is unbounded below and has exactly 2^d full-rank critical points, one in each diagonal sign chamber. Every critical point is diagonal and hyperbolic. Its unstable dimension is binom d2 and its stable dimension is d(d+1)/2. We also prove that a full-rank trajectory cannot converge to a finite rank-deficient matrix. Consequently, almost every full-rank initial condition has an unbounded forward orbit. Thus finite-temperature fiber entropy does not provide an equilibrium selection principle for diagonal completion.
ER -