Projected Energy Matching for Generative 3D Priors
Abstract
Transport-based generative models, which learn a time-dependent vector field that moves noise to data, have become a dominant paradigm. However, these models typically do not explicitly encode the data distribution. Energy-based models (EBMs) instead represent the data distribution explicitly through a scalar energy landscape, which Energy Matching learns by combining transport learning with contrastive refinement. Its transport objective, however, fits energy gradients to stochastic targets, whose variance degrades the training signal at scale. We introduce Projected Energy Matching, which learns this landscape through a more stable route: we first train a time-independent transport teacher, then freeze it and fit the negative energy gradient to its predicted velocities. This projection replaces noisy transport targets with deterministic supervision, while contrastive refinement shapes the landscape near the data manifold. On CIFAR-10, gradient-noise analysis reveals a cleaner training signal, accompanied by faster convergence than Energy Matching at matched, teacher-free training budgets. In the latent space of CT volumes, our method enables 3D CT generation and achieves better FID scores than flow models. The learned scalar potential serves as a zero-shot prior for the ill-posed inverse problem of sparse-view cone-beam CT reconstruction. By making explicit energy landscapes practical at volumetric scale, this work opens a path to wider adoption of energy-based formulations, bringing their flexibility to high-dimensional generation and inverse problems.
Related
- Flow Matching with In-Context Priors for Out-of-Distribution Brain Dynamics
- Do Generative Priors Align with Human Naturalness Perception?
- Generalizable single-cell perturbation response prediction using energy-guided flow matching
- A generative framework for estimating brain energy landscapes from continuous neuroimaging observations.
- The Ocean Microbiomics Database reveals thermal niches and projected pressure on marine microbes in a warming ocean
BibTeX
@misc{barco2026projected,
title = {{Projected Energy Matching for Generative 3D Priors}},
author = {Daniel Barco and Michal Balcerak and Suprosanna Shit and Chinmay Prabhakar and Philipp Denzel and Bjoern Menze and Frank-Peter Schilling},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2607.07749},
eprint = {2607.07749},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2607.07749},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Projected Energy Matching for Generative 3D Priors
AU - Daniel Barco
AU - Michal Balcerak
AU - Suprosanna Shit
AU - Chinmay Prabhakar
AU - Philipp Denzel
AU - Bjoern Menze
AU - Frank-Peter Schilling
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2607.07749
UR - https://arxiv.org/abs/2607.07749
AB - Transport-based generative models, which learn a time-dependent vector field that moves noise to data, have become a dominant paradigm. However, these models typically do not explicitly encode the data distribution. Energy-based models (EBMs) instead represent the data distribution explicitly through a scalar energy landscape, which Energy Matching learns by combining transport learning with contrastive refinement. Its transport objective, however, fits energy gradients to stochastic targets, whose variance degrades the training signal at scale. We introduce Projected Energy Matching, which learns this landscape through a more stable route: we first train a time-independent transport teacher, then freeze it and fit the negative energy gradient to its predicted velocities. This projection replaces noisy transport targets with deterministic supervision, while contrastive refinement shapes the landscape near the data manifold. On CIFAR-10, gradient-noise analysis reveals a cleaner training signal, accompanied by faster convergence than Energy Matching at matched, teacher-free training budgets. In the latent space of CT volumes, our method enables 3D CT generation and achieves better FID scores than flow models. The learned scalar potential serves as a zero-shot prior for the ill-posed inverse problem of sparse-view cone-beam CT reconstruction. By making explicit energy landscapes practical at volumetric scale, this work opens a path to wider adoption of energy-based formulations, bringing their flexibility to high-dimensional generation and inverse problems.
ER -