LocalProp: Neuro-Localized Memory-Efficient Backpropagation

Abstract

The current deep learning training paradigm employs end-to-end backpropagation, regardless of the training stage, i.e. pre-training or fine-tuning. However, backpropagating through the entire model is neither biologically plausible nor memory efficient, since learning inside the brain is highly localized. Therefore, we propose LocalProp, a training procedure that locally updates the weights of a model. Our neuro-localized weight updates follow the "pre-training then fine-tuning" paradigm, where the pre-training is based on I-JEPA. After locally updating the weights, a pruning operation is performed, followed by a short final fine-tuning phase. Pruning helps by sending the learning signal from higher blocks to lower blocks. We perform experiments on several datasets, including large-scale benchmarks such as ImageNet, and empirically show that LocalProp reaches good performance at a fraction of GPU peak memory. By varying the number of jointly optimized blocks, we identify gradient-propagation span as a practical control over the accuracy-memory trade-off.

Localized charges, reset noise, and boundary memory in matrix-product-conserving quantum chains
Ron Rubin
arXiv19 Sep 2026PreprintarXiv licenceFull textMathematical physics Quantum physics & quantum informationarXiv
Jacobian-Free Unrolling: memory efficient unrolled network for 3D SPECT Reconstruction
Corentin Constanza, Théo Kaprelian, Ane Etxebeste +1
IEEE Transactions on Radiation and Plasma Medical Sciences2 Nov 2026Peer-reviewedPublisher termsFull textSystems & synthetic biology Optimisation & controlHAL
PowerStep: Memory-Efficient Adaptive Optimization via ℓ_p-Norm Steepest Descent
Yao Lu, Dengdong Fan, Shixun Zhang +1
arXiv29 Sep 2026PreprintCC BY 4.0Full textNumerical analysis & scientific computing Optimisation & controlarXiv
Finite-Memory Control of POMDPs: Fundamental Limits and Efficient Design
Emrecan Kutay, Atilla Eryilmaz, Ness Shroff
arXiv2 Nov 2026PreprintCC BY 4.0Full textGeneral mathematicsarXiv
Low-Rank Friction for Memory-Efficient Transformer Pretraining
Rajit Rajpal, Benedict Leimkuhler
arXiv28 Sep 2026PreprintCC BY 4.0Full textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{grigore2026localprop,
  title = {{LocalProp: Neuro-Localized Memory-Efficient Backpropagation}},
  author = {Diana-Nicoleta Grigore and Iuliana Georgescu and Radu Tudor Ionescu},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.32517},
  eprint = {2609.32517},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.32517},
}

RIS

TY  - UNPB
TI  - LocalProp: Neuro-Localized Memory-Efficient Backpropagation
AU  - Diana-Nicoleta Grigore
AU  - Iuliana Georgescu
AU  - Radu Tudor Ionescu
PY  - 2026
DA  - 2026/09/26
PB  - arXiv
DO  - 10.48550/arxiv.2609.32517
UR  - https://arxiv.org/abs/2609.32517
AB  - The current deep learning training paradigm employs end-to-end backpropagation, regardless of the training stage, i.e. pre-training or fine-tuning. However, backpropagating through the entire model is neither biologically plausible nor memory efficient, since learning inside the brain is highly localized. Therefore, we propose LocalProp, a training procedure that locally updates the weights of a model. Our neuro-localized weight updates follow the "pre-training then fine-tuning" paradigm, where the pre-training is based on I-JEPA. After locally updating the weights, a pruning operation is performed, followed by a short final fine-tuning phase. Pruning helps by sending the learning signal from higher blocks to lower blocks. We perform experiments on several datasets, including large-scale benchmarks such as ImageNet, and empirically show that LocalProp reaches good performance at a fraction of GPU peak memory. By varying the number of jointly optimized blocks, we identify gradient-propagation span as a practical control over the accuracy-memory trade-off.
ER  -

Abstract

Related

Cite