The critical slowing down in training diffusion models
Abstract
Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently enabled major advances, their behavior remains poorly understood, with limited theoretical control over when and why they succeed. Here we provide such insight for diffusion models---a class of generative schemes highly effective in practice---by analyzing their application to the O(n) model of statistical field theory in the Gaussian limit n → ∞. In this analytically tractable setting, we show that training a score model with a one-layer network architecture matching the exact solution exhibits a form of critical slowing down in parameter learning. This slowing down also impacts the generation process, indicating that the well-known difficulties of sampling near criticality persist even for learned generative models. To overcome this bottleneck, we consider the power of architectural depth. We find that using a two-layer architecture drastically reduces the critical slowing down, with the training time scaling logarithmically rather than quadratically with system size. Using a Fourier implementation of the architecture, we further show that this acceleration in training time can be achieved without drastically increasing operational complexity. Taken together, these results demonstrate that diffusion models can overcome the critical slowing down through appropriate architectural design, and establish a controlled framework for understanding and improving learned sampling methods in statistical physics and beyond.
Related
- First-Order Transition to Chaos with Critical Slowing Down
- Overcoming critical slowing down in frustrated spin systems by learned multiscale sampling
- Stochastic model of elongation-shortening-switching kinetics of a polymeric filament: Critical slowing down of first-passage and finite-size regularization
- Beyond Critical Slowing Down: Slow Modes, Extreme Tails, and Field Decoherence in Tipping Transitions
- Systematic errors from scalar slowing-down models in transient SERF magnetometry
BibTeX
@misc{bono2026critical,
title = {{The critical slowing down in training diffusion models}},
author = {Luca Maria Del Bono and Giulio Biroli and Patrick Charbonneau and Marylou Gabrié},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2605.12597},
eprint = {2605.12597},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.12597},
}
RIS
TY - UNPB
TI - The critical slowing down in training diffusion models
AU - Luca Maria Del Bono
AU - Giulio Biroli
AU - Patrick Charbonneau
AU - Marylou Gabrié
PY - 2026
DA - 2026/09/18
PB - arXiv
DO - 10.48550/arxiv.2605.12597
UR - https://arxiv.org/abs/2605.12597
AB - Computational sampling has been central to the sciences since the mid-20th century. While machine-learning-based approaches have recently enabled major advances, their behavior remains poorly understood, with limited theoretical control over when and why they succeed. Here we provide such insight for diffusion models---a class of generative schemes highly effective in practice---by analyzing their application to the O(n) model of statistical field theory in the Gaussian limit n → ∞. In this analytically tractable setting, we show that training a score model with a one-layer network architecture matching the exact solution exhibits a form of critical slowing down in parameter learning. This slowing down also impacts the generation process, indicating that the well-known difficulties of sampling near criticality persist even for learned generative models. To overcome this bottleneck, we consider the power of architectural depth. We find that using a two-layer architecture drastically reduces the critical slowing down, with the training time scaling logarithmically rather than quadratically with system size. Using a Fourier implementation of the architecture, we further show that this acceleration in training time can be achieved without drastically increasing operational complexity. Taken together, these results demonstrate that diffusion models can overcome the critical slowing down through appropriate architectural design, and establish a controlled framework for understanding and improving learned sampling methods in statistical physics and beyond.
ER -