Valid Early Detection of Memorization in Diffusion Models
Abstract
Diffusion models can sometimes reproduce examples from their training data, a phenomenon known as memorization. Existing memorization detection methods for unconditional diffusion models typically operate at a pre-selected timestep and therefore cannot adapt the detection time to the evolving denoising trajectory. Sequential monitoring enables adaptive detection but repeated testing across timesteps complicates trajectory-level false-alarm control. We study two ways of combining split conformal calibration with sequential monitoring for prompt-free memorization detection: per-step Bonferroni calibration and direct trajectory-level calibration of the maximum memorization score. The former controls repeated testing by splitting the false-alarm budget across timesteps, whereas the latter calibrates the event of ever crossing a threshold directly through the trajectory-level maximum. We evaluate this formulation on an unconditional CIFAR-10 EDM using two complementary per-step statistics based on geometry and representation spikiness. Experiments show that sequential monitoring improves the power-timing tradeoff over fixed-time detection. Compared with Bonferroni calibration, trajectory-level calibration achieves comparable detection power while using the false-alarm budget less conservatively and enabling earlier detection at higher noise levels.
Related
- Loss Curvature as a Sampling-Free Proxy for Memorization in Diffusion Models
- Anytime-valid detection of LLM weight exfiltration
- Exact Multiple Change-Point Detection Via Smallest Valid Partitioning
- Anytime-Valid Distribution Shift Detection via Predictive Rank Martingales
- Posterior Frequencies and the Cost of Not Fitting: A Correction to a Memorization Lower Bound
BibTeX
@misc{jing2026valid,
title = {{Valid Early Detection of Memorization in Diffusion Models}},
author = {Yixiao Jing and Chia-Yu Hsu and Milad Hoseinpour and Zekai Zhang},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.34617},
eprint = {2610.34617},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.34617},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Valid Early Detection of Memorization in Diffusion Models
AU - Yixiao Jing
AU - Chia-Yu Hsu
AU - Milad Hoseinpour
AU - Zekai Zhang
PY - 2026
DA - 2026/10/25
PB - arXiv
DO - 10.48550/arxiv.2610.34617
UR - https://arxiv.org/abs/2610.34617
AB - Diffusion models can sometimes reproduce examples from their training data, a phenomenon known as memorization. Existing memorization detection methods for unconditional diffusion models typically operate at a pre-selected timestep and therefore cannot adapt the detection time to the evolving denoising trajectory. Sequential monitoring enables adaptive detection but repeated testing across timesteps complicates trajectory-level false-alarm control. We study two ways of combining split conformal calibration with sequential monitoring for prompt-free memorization detection: per-step Bonferroni calibration and direct trajectory-level calibration of the maximum memorization score. The former controls repeated testing by splitting the false-alarm budget across timesteps, whereas the latter calibrates the event of ever crossing a threshold directly through the trajectory-level maximum. We evaluate this formulation on an unconditional CIFAR-10 EDM using two complementary per-step statistics based on geometry and representation spikiness. Experiments show that sequential monitoring improves the power-timing tradeoff over fixed-time detection. Compared with Bonferroni calibration, trajectory-level calibration achieves comparable detection power while using the false-alarm budget less conservatively and enabling earlier detection at higher noise levels.
ER -