Noise in Diffusion Models Is a Learnable Input
Abstract
Stochastic learning objectives are typically written as expectations over abstract random variables. Actual training, however, uses concrete random inputs that enter both the realized loss and its gradient. Structure in these inputs that is accessible to the learning system can therefore be learned and exploited. Much prior structured-noise work asks how noise should be distributed or designed; we instead ask what structure in the concrete realized randomness becomes exploitable by the learner. We develop this general view and analyze its mechanism in diffusion models: in noise prediction, clean data and realized noise jointly form the noisy input, so the model can improve prediction by learning clean-data regularities or exploiting structure introduced through the noise, and the two routes can interact. Using pseudorandom streams as controlled, reproducible instances of structured noise, we provide mechanistic evidence on MNIST and CIFAR-10: random-role ablations localize the dominant effect to diffusion noise; in a diffusion probe, structured-noise training can reduce prediction loss below the IID reference, but replacing the test noise with IID reverses this advantage; and shuffling the same values largely removes the source-dependent loss reduction. This learned dependence can also affect generation. The same view offers a unified interpretation of data-dependent noise assignment, noise-based backdoors, and temporally correlated noise in video diffusion: although these methods introduce different structures, all alter what the model can exploit through noise and its interaction with clean-data learning. Our results indicate that diffusion noise is not merely a passive stochastic perturbation, but a learnable---and therefore potentially designable---input dimension.
Related
- Stochastic fractional diffusion equations with spatial Gaussian noise in the Stratonovich regime
- The Noise-Transmission Burden of Diffusion Network: Why Image Prediction Can Outperform Noise Prediction
- Large Deviations of Invariant Measures of Fractional Reaction-Diffusion Equations Driven by Multiplicative Noise on R^n
- Training data attribution in diffusion models via mirrored unlearning and noise-consistent skew
- State-law dynamics of McKean-Vlasov stochastic reaction-diffusion equations on ℝn: pullback random attractors and zero-noise stability
BibTeX
@misc{deng2026noise,
title = {{Noise in Diffusion Models Is a Learnable Input}},
author = {Shengzhi Deng and Chenqi Ye and Yanze Guo},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2608.02575},
eprint = {2608.02575},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.02575},
}
RIS
TY - UNPB
TI - Noise in Diffusion Models Is a Learnable Input
AU - Shengzhi Deng
AU - Chenqi Ye
AU - Yanze Guo
PY - 2026
DA - 2026/09/08
PB - arXiv
DO - 10.48550/arxiv.2608.02575
UR - https://arxiv.org/abs/2608.02575
AB - Stochastic learning objectives are typically written as expectations over abstract random variables. Actual training, however, uses concrete random inputs that enter both the realized loss and its gradient. Structure in these inputs that is accessible to the learning system can therefore be learned and exploited. Much prior structured-noise work asks how noise should be distributed or designed; we instead ask what structure in the concrete realized randomness becomes exploitable by the learner. We develop this general view and analyze its mechanism in diffusion models: in noise prediction, clean data and realized noise jointly form the noisy input, so the model can improve prediction by learning clean-data regularities or exploiting structure introduced through the noise, and the two routes can interact. Using pseudorandom streams as controlled, reproducible instances of structured noise, we provide mechanistic evidence on MNIST and CIFAR-10: random-role ablations localize the dominant effect to diffusion noise; in a diffusion probe, structured-noise training can reduce prediction loss below the IID reference, but replacing the test noise with IID reverses this advantage; and shuffling the same values largely removes the source-dependent loss reduction. This learned dependence can also affect generation. The same view offers a unified interpretation of data-dependent noise assignment, noise-based backdoors, and temporally correlated noise in video diffusion: although these methods introduce different structures, all alter what the model can exploit through noise and its interaction with clean-data learning. Our results indicate that diffusion noise is not merely a passive stochastic perturbation, but a learnable---and therefore potentially designable---input dimension.
ER -