Exact Realization of Deep ReLU Computation in a Microscopic Statistical-Mechanical System
Abstract
Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in β→+∞ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.
Related
- Tag-system computation using only powers and principal logarithms
- Certified computation of Buehler confidence limits for system reliability
- Microstructural and physicochemical characterization, mechanical and durability performance with statistical evaluation of montmorillonite-impregnated burnt red soil bricks incorporating pyrolyzed coffee and tobacco biochars
- GradAgent: A Knowledge-Guided Multi-Agent System for Structure-Preserving Gradient-Flow Computation with an Application to Multicomponent Vesicle Dynamics
- Bayesian statistical inverse problems for a coupled Fokker-Planck-Darcy system
BibTeX
@misc{li2026exact,
title = {{Exact Realization of Deep ReLU Computation in a Microscopic Statistical-Mechanical System}},
author = {Junxu Li},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.22965},
eprint = {2609.22965},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22965},
}
RIS
TY - UNPB
TI - Exact Realization of Deep ReLU Computation in a Microscopic Statistical-Mechanical System
AU - Junxu Li
PY - 2026
DA - 2026/09/19
PB - arXiv
DO - 10.48550/arxiv.2609.22965
UR - https://arxiv.org/abs/2609.22965
AB - Understanding the physical origin of the piecewise-linear computation in deep rectified linear unit (ReLU) networks remains a fundamental challenge. Here we establish an exact statistical-mechanical realization of arbitrary ReLU networks. Starting from a microscopic configuration space and its state multiplicities, we construct a partition function without prescribing a neural-network activation function. The resulting system admits equivalent descriptions in terms of cascaded quantum operations with post selection, or fermionic transport model. We rigorously prove that in β→+∞ limit, the thermodynamic observables of this system exactly reproduce the hidden states, outputs, and loss function of an arbitrary deep ReLU network. Crucially, we demonstrate that the piecewise-linear behavior of the ReLU network emerges from a first-order phase transition in the microscopic system, where the discontinuities precisely coincide with the boundaries of linear regions of ReLU network. This work establishes an exact physical foundation for deep neural computation and provides a novel statistical-mechanical perspective on neural architecture design.
ER -