Memorization Capacity of Invariant Convolutional Neural Nets
Abstract
Neural networks are powerful memorizers which are often able to interpolate their training data even when the labels have been randomized. While there exists an extensive literature on the memorization capacity of fully connected networks, in practice more specific architectures with problem-dependent properties are preferred, such as convolutional networks for image classification. In this paper we consider convolutional networks which are invariant with respect to a given arbitrary finite group of orthogonal transformations, which may include translations, reflections, and/or rotations of images and invariances of higher order tensors. We prove upper bounds on the size of an invariant convolutional neural network with threshold activations that is required to interpolate a given dataset with two classes. Our bounds are formulated in terms of a complexity measure that takes the invariant structure into account and is instance-specific, i.e., it is related to the mutual arrangement of the classes in the given dataset and their geometric properties rather than the number of samples. Our proof is constructive: for given input data, instances of an interpolating network can be computed via a randomized algorithm with high probability. In numerical experiments we investigate the practical effectiveness of this algorithm.
Related
- Estimating "Realized" Skewness using Convolutional Neural Network
- Stochastic dynamics generation of directed percolation using a two-channel convolutional neural network
- Initial condition recovery in nonlinear damped viscous photoacoustic tomography using a convolutional neural network-guided gradient-free optimization framework
- The role of locality and weight sharing in convolutional neural networks
- Symplectic-inspired neural operators for discretization-invariant light propagation with exact power conservation
BibTeX
@misc{dirksen2026memorization,
title = {{Memorization Capacity of Invariant Convolutional Neural Nets}},
author = {Sjoerd Dirksen and Patrick Finke and Paul Geuchen},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.38938},
eprint = {2610.38938},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.38938},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Memorization Capacity of Invariant Convolutional Neural Nets
AU - Sjoerd Dirksen
AU - Patrick Finke
AU - Paul Geuchen
PY - 2026
DA - 2026/10/28
PB - arXiv
DO - 10.48550/arxiv.2610.38938
UR - https://arxiv.org/abs/2610.38938
AB - Neural networks are powerful memorizers which are often able to interpolate their training data even when the labels have been randomized. While there exists an extensive literature on the memorization capacity of fully connected networks, in practice more specific architectures with problem-dependent properties are preferred, such as convolutional networks for image classification. In this paper we consider convolutional networks which are invariant with respect to a given arbitrary finite group of orthogonal transformations, which may include translations, reflections, and/or rotations of images and invariances of higher order tensors. We prove upper bounds on the size of an invariant convolutional neural network with threshold activations that is required to interpolate a given dataset with two classes. Our bounds are formulated in terms of a complexity measure that takes the invariant structure into account and is instance-specific, i.e., it is related to the mutual arrangement of the classes in the given dataset and their geometric properties rather than the number of samples. Our proof is constructive: for given input data, instances of an interpolating network can be computed via a randomized algorithm with high probability. In numerical experiments we investigate the practical effectiveness of this algorithm.
ER -