Annealed correlation and quenched noise sensitivity in biased deep neural networks
Abstract
In earlier work by Jonasson, Steif and Zetterqvist (2023), fully connected deep neural networks with layers of width n, i.i.d.\ standard Gaussian weights, sign activations and no bias terms were shown to be annealed and quenched noise sensitive with respect to any joint probability distribution ℚ(n) on the input ω and its perturbation η satisfying a mild positive-correlation condition and ℚ(n)(η ∈ {ω,−ω}) → 0. Here we add independent Gaussian biases of the natural order and show that quenched noise sensitivity is retained over a broad range of depths, indeed under the slightly weaker assumption that ℚ(n)(ω = η) → 0. Thus, subject to the positive-correlation condition, quenched noise sensitivity holds under the minimal requirement that the perturbed input is asymptotically unlikely to coincide with the original input. At the same time, the annealed covariance remains bounded away from zero and is entirely accounted for by fluctuations in the network-dependent output means.
Related
- A second-moment proof of quenched equals annealed for the Potts model on random regular graphs
- Universality of Annealed Complexity in the Spherical Pure p-Spin Model
- Annealed Survival Probability of Random Walk in an Inhomogeneous Poisson Environment of Mobile Traps
- Small-time annealed large deviations principle for one-dimensional diffusions in a random environment
- Averaged Annealed Markov Chains Achieve the Algorithmic Threshold for Tensor PCA, Part I: Glauber Dynamics
BibTeX
@misc{jonasson2026annealed,
title = {{Annealed correlation and quenched noise sensitivity in biased deep neural networks}},
author = {Johan Jonasson},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.27983},
eprint = {2610.27983},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.27983},
}
RIS
TY - UNPB
TI - Annealed correlation and quenched noise sensitivity in biased deep neural networks
AU - Johan Jonasson
PY - 2026
DA - 2026/10/20
PB - arXiv
DO - 10.48550/arxiv.2610.27983
UR - https://arxiv.org/abs/2610.27983
AB - In earlier work by Jonasson, Steif and Zetterqvist (2023), fully connected deep neural networks with layers of width n, i.i.d.\ standard Gaussian weights, sign activations and no bias terms were shown to be annealed and quenched noise sensitive with respect to any joint probability distribution ℚ^(n) on the input ω and its perturbation η satisfying a mild positive-correlation condition and ℚ^(n)(η ∈ {ω,−ω}) → 0. Here we add independent Gaussian biases of the natural order and show that quenched noise sensitivity is retained over a broad range of depths, indeed under the slightly weaker assumption that ℚ^(n)(ω = η) → 0. Thus, subject to the positive-correlation condition, quenched noise sensitivity holds under the minimal requirement that the perturbed input is asymptotically unlikely to coincide with the original input. At the same time, the annealed covariance remains bounded away from zero and is entirely accounted for by fluctuations in the network-dependent output means.
ER -