Capacity of the Hopfield-Hebbian dynamics with correlated patterns
Abstract
Allowing for small fraction of errors in retrieved stored binary patterns, Hopfield-Hebbian neural network (NN) associative memory dynamics [14] achieves linearly scaled capacity. The famous proportionality constant αc≈ 0.14 is obtained in a large dimensional scenario as the ratio of the number of reliably stored patterns m and the patterns' length n under the assumption that uniformly randomly chosen binary patterns are \emph{independent} among themselves. While such an idealized scenario is now well understood, more practical scenarios with correlated/noisy datasets remaining challenging even for classical statistical mechanics replica approaches. We consider precisely such practical settings where each \emph{archetype} pattern is replaced with M correlated \emph{examples} with parameter r governing the quality of correlation. We develop a powerful Random Duality Theory (RDT) based framework to analyze correlated data and already on the first level of RDT lifting obtain capacity characterizations that remarkably well match behavior exhibited through simulated dynamics. Moreover, characterizations explicitly feature correlation parameters M and r, which allows to precisely determine their effects on phase-transitioning behavior of the underlying dynamics. Our results are also a well-fitting complement to the existing literature. Namely, to circumvent challenging statistical mechanics analyses, an alternative combination of spectral and replica methods is considered in [4]. We confirm that [4]'s elegant spectral characterizations are excellent approximations in the low correlated regime. At the same time, we also close the gap in the regimes with higher correlation levels where spectral methods deviate from the practical dynamics performance.
Related
- The solution of the Hopfield model with Gaussian patterns
- Free energy and fluctuations of generalized bipartite Sherrington-Kirkpatrick models and generalized Hopfield models
- Free energy of the Hopfield model
- Periodic Solutions and Exponential Attractivity in Hopfield-Type Neural Networks with Generalized Piecewise Constant Arguments: Impulsive and Nonimpulsive Cases
- Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks
BibTeX
@misc{stojnic2026capacity,
title = {{Capacity of the Hopfield-Hebbian dynamics with correlated patterns}},
author = {Mihailo Stojnic},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.38437},
eprint = {2610.38437},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.38437},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Capacity of the Hopfield-Hebbian dynamics with correlated patterns
AU - Mihailo Stojnic
PY - 2026
DA - 2026/10/28
PB - arXiv
DO - 10.48550/arxiv.2610.38437
UR - https://arxiv.org/abs/2610.38437
AB - Allowing for small fraction of errors in retrieved stored binary patterns, Hopfield-Hebbian neural network (NN) associative memory dynamics [14] achieves linearly scaled capacity. The famous proportionality constant α_c≈ 0.14 is obtained in a large dimensional scenario as the ratio of the number of reliably stored patterns m and the patterns' length n under the assumption that uniformly randomly chosen binary patterns are \emph{independent} among themselves. While such an idealized scenario is now well understood, more practical scenarios with correlated/noisy datasets remaining challenging even for classical statistical mechanics replica approaches. We consider precisely such practical settings where each \emph{archetype} pattern is replaced with M correlated \emph{examples} with parameter r governing the quality of correlation. We develop a powerful Random Duality Theory (RDT) based framework to analyze correlated data and already on the first level of RDT lifting obtain capacity characterizations that remarkably well match behavior exhibited through simulated dynamics. Moreover, characterizations explicitly feature correlation parameters M and r, which allows to precisely determine their effects on phase-transitioning behavior of the underlying dynamics. Our results are also a well-fitting complement to the existing literature. Namely, to circumvent challenging statistical mechanics analyses, an alternative combination of spectral and replica methods is considered in [4]. We confirm that [4]'s elegant spectral characterizations are excellent approximations in the low correlated regime. At the same time, we also close the gap in the regimes with higher correlation levels where spectral methods deviate from the practical dynamics performance.
ER -