Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks
Abstract
Equilibrium Propagation (EqProp) offers a biologically grounded and hardware-aligned alternative to backpropagation for training Hopfield Networks. However, its behavior in continual learning settings remains largely unexplored. This work confronts catastrophic forgetting in EqProp-trained Hopfield Networks from the inside out, eliminating the need for external data buffers or architectural overhead that standard Experience Replay \textbf{(ER)} and Generative Replay \textbf{(Gen-Replay)} inevitably require. We introduce an Energy-Based Generative Replay strategy \textbf{(EB-Gen)} which exploits the intrinsic generative dynamics of Hopfield Networks, allowing a single model to act as both classifier and replay samples generator. Samples capturing past task knowledge are generated by relaxing a frozen copy of the network, enabling rehearsal solely through internally generated pseudo-data. We evaluate the proposed method in the Class-Incremental setting, using the split versions of the MNIST and CIFAR10 benchmarks and compare it with FineTune, Experience Replay, Generative Replay, Sleep-Like Replay Consolidation (SRC) and Joint training. Our results show that EB-Gen significantly improves stability while preserving plasticity, achieving a 75\% and 64\% average accuracy on Split MNIST and Split CIFAR10 respectively. These findings show that Equilibrium Propagation can be adapted to alleviate catastrophic forgetting without forgoing the traits which made it compelling in the first place.
Related
- Reliable Replay through Spatial Coherence in Online Continual Learning
- Fast Weight Attention for Continual Learning
- ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
- A Dynamical Theory of LoRA in Continual Learning
- CARE-MoE: Conflict-Aware Routing-Guided Expert Mixture-of-Experts for Continual Learning
BibTeX
@misc{cook2026generative,
title = {{Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks}},
author = {Victor Cook and Gilles Sassatelli and Thierry Gil and Mohamed Watfa and Marina Reyboz},
howpublished = {HAL},
year = {2026},
month = may,
eprint = {2610.37139},
archivePrefix = {arXiv},
url = {https://hal-lirmm.ccsd.cnrs.fr/lirmm-05525004v3},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Generative Replay for Equilibrium Propagation: Toward Edge-Friendly Continual Learning in Hopfield Networks
AU - Victor Cook
AU - Gilles Sassatelli
AU - Thierry Gil
AU - Mohamed Watfa
AU - Marina Reyboz
PY - 2026
DA - 2026/05/30
PB - HAL
UR - https://hal-lirmm.ccsd.cnrs.fr/lirmm-05525004v3
AB - Equilibrium Propagation (EqProp) offers a biologically grounded and hardware-aligned alternative to backpropagation for training Hopfield Networks. However, its behavior in continual learning settings remains largely unexplored. This work confronts catastrophic forgetting in EqProp-trained Hopfield Networks from the inside out, eliminating the need for external data buffers or architectural overhead that standard Experience Replay \textbf{(ER)} and Generative Replay \textbf{(Gen-Replay)} inevitably require. We introduce an Energy-Based Generative Replay strategy \textbf{(EB-Gen)} which exploits the intrinsic generative dynamics of Hopfield Networks, allowing a single model to act as both classifier and replay samples generator. Samples capturing past task knowledge are generated by relaxing a frozen copy of the network, enabling rehearsal solely through internally generated pseudo-data. We evaluate the proposed method in the Class-Incremental setting, using the split versions of the MNIST and CIFAR10 benchmarks and compare it with FineTune, Experience Replay, Generative Replay, Sleep-Like Replay Consolidation (SRC) and Joint training. Our results show that EB-Gen significantly improves stability while preserving plasticity, achieving a 75\% and 64\% average accuracy on Split MNIST and Split CIFAR10 respectively. These findings show that Equilibrium Propagation can be adapted to alleviate catastrophic forgetting without forgoing the traits which made it compelling in the first place.
ER -