DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models
Abstract
Parameter compression of class-conditional diffusion models exposes a structural limitation in output-level distillation: supervising only the guided output leaves the two score branches non-identifiable, so the classifier-free guidance gap is not determined in the student. The zero-loss set admits degenerate solutions in which both branches collapse toward identical predictions, and guidance loses its effect at inference despite low training loss. For each fixed input the objective is exactly flat in a branch-output direction at every residual value, so the ambiguity is a property of the output-level objective rather than a poor local minimum. This paper introduces DASH, which supervises the conditional and unconditional branches independently. An anchor term regularises the conditional prediction toward ground-truth noise, and the teacher's final learned per-timestep curriculum transfers into the student as a frozen prior. Across CIFAR-10, CIFAR-100, and class-conditional ImageNet-64, a more than 5× compressed student stays within four FID points of its teacher and recovers at least 89% of the teacher's guidance-gap magnitude, where no other two-branch baseline with defined calibration exceeds 82%. Ablation isolates unconditional supervision as the term that separates this formulation from outputlevel distillation. The account is quantitative: the null space fixes the ratio between a composite student's two branch errors, and the measured ratios approximately match that prediction on all three datasets.
Related
- Joint Branch-Space Transform Coding for Diffusion Activation Quantization with Classifier-Free Guidance
- Diffusion Distillation for Efficient Weather Ensembles
- Distribution Matching Distillation for Continuous Diffusion Language Models
- Steepest Guidance: A Practical and Principled Approach to Inference-Time Alignment of Flow and Diffusion-based Models
- Internal Guidance as a Sampling Method for Tilted Diffusion Models
BibTeX
@misc{shafi2026dash,
title = {{DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models}},
author = {Abdullah Al Shafi and Kazi Saeed Alam and Sk Imran Hossain and Engelbert Mephu Nguifo},
howpublished = {HAL},
year = {2026},
month = aug,
eprint = {2606.00798},
archivePrefix = {arXiv},
url = {https://hal.science/hal-05836726v1},
note = {CC BY-NC 4.0},
}
RIS
TY - UNPB
TI - DASH: Dual-Branch Score Distillation for Guidance-Calibrated Compact Diffusion Models
AU - Abdullah Al Shafi
AU - Kazi Saeed Alam
AU - Sk Imran Hossain
AU - Engelbert Mephu Nguifo
PY - 2026
DA - 2026/08/29
PB - HAL
UR - https://hal.science/hal-05836726v1
AB - Parameter compression of class-conditional diffusion models exposes a structural limitation in output-level distillation: supervising only the guided output leaves the two score branches non-identifiable, so the classifier-free guidance gap is not determined in the student. The zero-loss set admits degenerate solutions in which both branches collapse toward identical predictions, and guidance loses its effect at inference despite low training loss. For each fixed input the objective is exactly flat in a branch-output direction at every residual value, so the ambiguity is a property of the output-level objective rather than a poor local minimum. This paper introduces DASH, which supervises the conditional and unconditional branches independently. An anchor term regularises the conditional prediction toward ground-truth noise, and the teacher's final learned per-timestep curriculum transfers into the student as a frozen prior. Across CIFAR-10, CIFAR-100, and class-conditional ImageNet-64, a more than 5× compressed student stays within four FID points of its teacher and recovers at least 89% of the teacher's guidance-gap magnitude, where no other two-branch baseline with defined calibration exceeds 82%. Ablation isolates unconditional supervision as the term that separates this formulation from outputlevel distillation. The account is quantitative: the null space fixes the ratio between a composite student's two branch errors, and the measured ratios approximately match that prediction on all three datasets.
ER -