Double Descent and Malign Overfitting in Diffusion Models
Abstract
Conventional wisdom in deep learning holds that overparameterization---having more parameters p than training samples n---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might expect the same benign overfitting for diffusion models, whose training reduces to regression, i.e. to minimizing a quadratic score-matching loss. Yet the opposite is observed: overfitting here is catastrophic, driving the model into a memorization regime. We resolve this paradox by combining experiments on U-Nets trained on CelebA with a random-features model for which we derive closed-form learning curves. We show that with a fixed number m of noise realizations per training sample, an interpolation peak does occur, but at p∼ nm rather than at p∼ n as in standard regression. The rise of the test loss, however, sets in much earlier, at p∼ n, independently of m. This overfitting is malign because, although the implicit regularization of training is fully at work, it drives the model toward the empirical score, which memorizes the training set, rather than toward the true score. A bias-variance decomposition pinpoints the mechanism: the bias of the score estimator starts to grow at p∼ n; past the peak the variance decays, as in regression, whereas the bias keeps growing and both saturate at a large value. Since diffusion models are trained with m≫1, the peak is pushed to very large model sizes, and therefore sit on the rising branch that precedes it, where malign overfitting is already in play. Nevertheless, overparameterization remains beneficial when paired with regularization: in the random-features theory and in U-Net experiments, optimally regularized large models---via a ridge penalty or early stopping, respectively---outperform any unregularized models.
Related
- Shaping double descent in quantum extreme learning through model design
- Double descent is the principle of least action
- Balancing Expressivity and Overfitting in Quantum Gaussian Process Regression
- Replica Analysis of Overfitting in Vector Generalized Linear Models
- Camera-in-the-loop gradient descent hologram generation for Fraunhofer diffraction
BibTeX
@misc{urfin2026double,
title = {{Double Descent and Malign Overfitting in Diffusion Models}},
author = {Raphaël Urfin and Tony Bonnaire and Giulio Biroli and Marc Mézard},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.26392},
eprint = {2609.26392},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.26392},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Double Descent and Malign Overfitting in Diffusion Models
AU - Raphaël Urfin
AU - Tony Bonnaire
AU - Giulio Biroli
AU - Marc Mézard
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.26392
UR - https://arxiv.org/abs/2609.26392
AB - Conventional wisdom in deep learning holds that overparameterization---having more parameters p than training samples n---is benign: larger models generalize better and, even without regularization, interpolating models generalize well, the test error following a double-descent curve. One might expect the same benign overfitting for diffusion models, whose training reduces to regression, i.e. to minimizing a quadratic score-matching loss. Yet the opposite is observed: overfitting here is catastrophic, driving the model into a memorization regime. We resolve this paradox by combining experiments on U-Nets trained on CelebA with a random-features model for which we derive closed-form learning curves. We show that with a fixed number m of noise realizations per training sample, an interpolation peak does occur, but at p∼ nm rather than at p∼ n as in standard regression. The rise of the test loss, however, sets in much earlier, at p∼ n, independently of m. This overfitting is malign because, although the implicit regularization of training is fully at work, it drives the model toward the empirical score, which memorizes the training set, rather than toward the true score. A bias-variance decomposition pinpoints the mechanism: the bias of the score estimator starts to grow at p∼ n; past the peak the variance decays, as in regression, whereas the bias keeps growing and both saturate at a large value. Since diffusion models are trained with m≫1, the peak is pushed to very large model sizes, and therefore sit on the rising branch that precedes it, where malign overfitting is already in play. Nevertheless, overparameterization remains beneficial when paired with regularization: in the random-features theory and in U-Net experiments, optimally regularized large models---via a ridge penalty or early stopping, respectively---outperform any unregularized models.
ER -