Learning CNN Filters via Generalized Stein's Method
Abstract
Convolutional Neural Networks (CNNs) have undoubtedly revolutionized image data analysis and the field of computer vision. As the cornerstone of CNNs, the convolution operation enables the networks to extract abstract features and uncover hidden relationships in the image data. This paper considers the problem of estimating convolution filters from a statistical perspective using a classical tool --- Stein's formula. We first formulate CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors. Furthermore, we propose a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula. Theoretical analysis suggests that our estimation achieves an optimal convergence rate, comparable to that of generalized linear models where the link function is known. Extensive simulation studies and real data analyses demonstrate that our approach outperforms popular deep learning algorithms, such as Adam. Notably, our method extends beyond filter estimation and can be applied to nonlinear dimension reduction, providing a viable pathway for representation learning.
Related
- Conditioning on Subalgebras - Entropy Duality and Generalized Quantum Stein's Lemma in von Neumann Algebras
- Generalized Tan-Arlery-Rabaste-Lehmann-Ovarlez Lower Bound on Ambiguity Function of a Set of Sequences With Mismatched Filters
- Stein's generalized spherical maximal operator
- Approximating Distributions on Vertex Degrees in Random Hypergraphs via Stein's Method
- Stein's functional method for weakly dependent random variables
BibTeX
@misc{yang2026learning,
title = {{Learning CNN Filters via Generalized Stein's Method}},
author = {Guang Yang and Wei Shi and Yuan Cao and Long Feng},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.02875},
eprint = {2609.02875},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.02875},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning CNN Filters via Generalized Stein's Method
AU - Guang Yang
AU - Wei Shi
AU - Yuan Cao
AU - Long Feng
PY - 2026
DA - 2026/09/02
PB - arXiv
DO - 10.48550/arxiv.2609.02875
UR - https://arxiv.org/abs/2609.02875
AB - Convolutional Neural Networks (CNNs) have undoubtedly revolutionized image data analysis and the field of computer vision. As the cornerstone of CNNs, the convolution operation enables the networks to extract abstract features and uncover hidden relationships in the image data. This paper considers the problem of estimating convolution filters from a statistical perspective using a classical tool --- Stein's formula. We first formulate CNNs into a general index model with matrix-valued input, where convolution filters can be viewed as index vectors. Furthermore, we propose a novel singular value decomposition (SVD) based approach to accurately learn the convolution filters based on a generalized version of the first-order Stein's formula. Theoretical analysis suggests that our estimation achieves an optimal convergence rate, comparable to that of generalized linear models where the link function is known. Extensive simulation studies and real data analyses demonstrate that our approach outperforms popular deep learning algorithms, such as Adam. Notably, our method extends beyond filter estimation and can be applied to nonlinear dimension reduction, providing a viable pathway for representation learning.
ER -