Phases in a class of associative memories via hidden neurons
Abstract
Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class H, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class H splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.
Related
- Phase Transitions and Modulation in Dense Oscillatory Associative Memories
- Dense auto-hetero associative memories applied to noisy communication channels
- Factual recall in linear associative memories: sharp asymptotics and mechanistic insights
- Hidden kinetic correlations control collective phases of entropy-conditioned histories
- The Hilbert Basis Theorem for Non-Associative Skew Power Series Rings and Non-Associative Skew Laurent Series Rings over a Non-Associative Coefficient Ring
BibTeX
@misc{ota2026phases,
title = {{Phases in a class of associative memories via hidden neurons}},
author = {Toshihiro Ota and Masato Taki},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.10976},
eprint = {2609.10976},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.10976},
}
RIS
TY - UNPB
TI - Phases in a class of associative memories via hidden neurons
AU - Toshihiro Ota
AU - Masato Taki
PY - 2026
DA - 2026/09/10
PB - arXiv
DO - 10.48550/arxiv.2609.10976
UR - https://arxiv.org/abs/2609.10976
AB - Associative memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its retrieval update into softmax attention. The polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class H, whose model is fixed by a Lagrangian for each layer, taking the hidden neurons as the order parameter of retrieval. At polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible neurons, so their differences come from the visible entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes retrieval by quantized reassignments of attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at polynomial load and large-deviation at exponential load, and the class H splits retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.
ER -