Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers
Abstract
We propose a decoupled, layerwise method for structurally sparsifying the fully connected layers of pretrained neural networks. Rather than penalizing all layers jointly, our approach extracts shallow two-layer subnetworks, normalizes the inner weights, and applies a structured group penalty to the outer weight matrix of each block, processing layers sequentially to prune neurons and reduce the width of each layer. We prove that the constrained decoupled objective is equivalent at optimality to a specific joint penalty on the inner and outer weights, for any positively homogeneous activation, and thus admits a clean projected and proximal formulation. Our central finding is that this decoupled reformulation is more robust than coupled methods. In numerical experiments it provides a wider usable range of the regularization strength and a lower rate of catastrophic over-pruning than the tested joint baseline while maintaining comparable accuracy. We establish these properties in controlled classification and sparse-recovery studies, and examine their scope in a high-dimensional PINN stress test and in the feed-forward layers of OPT-1.3B.
Related
- Complex and Stable Complex Structures on Real Vector Bundles over Connected Sums of Quaternionic Projective Planes
- A toric Calabi-Yau counterexample to Schimpf's connected stable pairs pole conjecture
- Stable Complex Structures On Real Vector Bundles Over Connected Sums Of Quaternionic Projective Spaces
- Streaming Hypergraph Coloring via Palette Sparsification
- Convex Regression at Scale: Geometry-Preserving Constraint Sparsification
BibTeX
@misc{kulick2026layerwise,
title = {{Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers}},
author = {Charles Kulick and Armenak Petrosyan and Sui Tang},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.21126},
eprint = {2609.21126},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.21126},
}
RIS
TY - UNPB
TI - Layerwise Decoupling for Stable Structured Sparsification of Fully Connected Layers
AU - Charles Kulick
AU - Armenak Petrosyan
AU - Sui Tang
PY - 2026
DA - 2026/09/17
PB - arXiv
DO - 10.48550/arxiv.2609.21126
UR - https://arxiv.org/abs/2609.21126
AB - We propose a decoupled, layerwise method for structurally sparsifying the fully connected layers of pretrained neural networks. Rather than penalizing all layers jointly, our approach extracts shallow two-layer subnetworks, normalizes the inner weights, and applies a structured group penalty to the outer weight matrix of each block, processing layers sequentially to prune neurons and reduce the width of each layer. We prove that the constrained decoupled objective is equivalent at optimality to a specific joint penalty on the inner and outer weights, for any positively homogeneous activation, and thus admits a clean projected and proximal formulation. Our central finding is that this decoupled reformulation is more robust than coupled methods. In numerical experiments it provides a wider usable range of the regularization strength and a lower rate of catastrophic over-pruning than the tested joint baseline while maintaining comparable accuracy. We establish these properties in controlled classification and sparse-recovery studies, and examine their scope in a high-dimensional PINN stress test and in the feed-forward layers of OPT-1.3B.
ER -