Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements
Abstract
Feature caching accelerates diffusion sampling by replacing expensive network evaluations with predictions from previously computed activations. However, forecasts based only on past features cannot directly incorporate changes in the current denoising state. We investigate whether inexpensive, freshly computed features can serve as observations for correcting these predictions. We introduce an observation-corrected Koopman framework for accelerating frozen diffusion models. Using calibration trajectories, we identify finite-dimensional, time-dependent Koopman approximations that jointly describe the increments of shallow and deep network features. During accelerated sampling, these operators predict the evolution of expensive deep features, while innovations in the observed shallow features correct the predicted state. Periodic full evaluations refresh the observer, and all generative-model parameters remain unchanged. This formulation enables controlled comparisons of temporal prediction and observation correction. Across three 10,000-image runs per dataset, our method reduces paired Inception-feature MSE by 19.9% on CIFAR-10 and 11.9% on a ten-class ImageNet subset relative to channelwise affine prediction under the same four-partial-step schedule. Matched ablations attribute additional reductions of 4.54% and 4.67% to observation correction. The observer achieves 1.89× and 1.85× measured speedups over DDIM-50, supporting improved reference-sampler fidelity without retraining the denoiser.
Related
- Comparing Measurements of Pictorial Depth: Conditions for Placing Methods, Observers and Depiction Styles on One Scale
- Beam Search for Error-Correcting Codes via Conditional Diffusion
- Transport-Matched Penalties for Diffusion Synthetic Acceleration of Polytopic Discontinuous Galerkin Discretisations
- Uniform convergence of diffusion synthetic acceleration for heterogeneous slab transport
- Acceleration of Diffusion Language Model through Discrete Average Generator
BibTeX
@misc{bai2026koopman,
title = {{Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements}},
author = {Hanru Bai and Yuanchao Xu and Fengyi Li},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.10366},
eprint = {2610.10366},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.10366},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurements
AU - Hanru Bai
AU - Yuanchao Xu
AU - Fengyi Li
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.10366
UR - https://arxiv.org/abs/2610.10366
AB - Feature caching accelerates diffusion sampling by replacing expensive network evaluations with predictions from previously computed activations. However, forecasts based only on past features cannot directly incorporate changes in the current denoising state. We investigate whether inexpensive, freshly computed features can serve as observations for correcting these predictions. We introduce an observation-corrected Koopman framework for accelerating frozen diffusion models. Using calibration trajectories, we identify finite-dimensional, time-dependent Koopman approximations that jointly describe the increments of shallow and deep network features. During accelerated sampling, these operators predict the evolution of expensive deep features, while innovations in the observed shallow features correct the predicted state. Periodic full evaluations refresh the observer, and all generative-model parameters remain unchanged. This formulation enables controlled comparisons of temporal prediction and observation correction. Across three 10,000-image runs per dataset, our method reduces paired Inception-feature MSE by 19.9% on CIFAR-10 and 11.9% on a ten-class ImageNet subset relative to channelwise affine prediction under the same four-partial-step schedule. Matched ablations attribute additional reductions of 4.54% and 4.67% to observation correction. The observer achieves 1.89× and 1.85× measured speedups over DDIM-50, supporting improved reference-sampler fidelity without retraining the denoiser.
ER -