Shrinkage prior for implicit generative models
Abstract
Implicit generative models based on deep neural networks have revolutionized many scientific fields, from weather research to language generation. They successfully model complex random objects arising in real-world problems. In this work, we propose a sample-efficient training mechanism for these models. Our approach leverages regularized inference with shrinkage priors and generalized Bayes. In the large p (parameter dimension) and small n (sample size) setting, sparsity-inducing priors have been successfully used for linear regression when p ≫ n. However, their application to implicit generative models remains challenging due to the intractability of the likelihood function. To address this, we adapt a scoring rule posterior based on a strictly proper scoring rule, following the generalized Bayes framework. We then propose a block SGMCMC within Gibbs sampling mechanism. This enables efficient exploration of high-dimensional parameter spaces by identifying a sparse representation of neural implicit generative models. We demonstrate the performance of our proposed method through experiments on p ≫ n linear regression and an application of neural generative models.
Related
- Weak Factor Recovery in Sparse Bayesian Factor Models under Exchangeable Shrinkage Process Prior
- The Prior-Force Family of Implicit Denoiser Priors and Its Decisions in Linear Inverse Problems
- Controlling Dependence in Implicit Generative Models via Spread Mutual Information
- Eigenvector distributions and optimal shrinkage estimators for large covariance and precision matrices
- Pathwise Shrinkage and Probabilistic Calibration of the Multiplicative Half-Cauchy Process
BibTeX
@misc{sinharoy2026shrinkage,
title = {{Shrinkage prior for implicit generative models}},
author = {Shreya Sinha-Roy and Christian P. Robert and Ritabrata Dutta},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.05241},
eprint = {2611.05241},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.05241},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Shrinkage prior for implicit generative models
AU - Shreya Sinha-Roy
AU - Christian P. Robert
AU - Ritabrata Dutta
PY - 2026
DA - 2026/11/03
PB - arXiv
DO - 10.48550/arxiv.2611.05241
UR - https://arxiv.org/abs/2611.05241
AB - Implicit generative models based on deep neural networks have revolutionized many scientific fields, from weather research to language generation. They successfully model complex random objects arising in real-world problems. In this work, we propose a sample-efficient training mechanism for these models. Our approach leverages regularized inference with shrinkage priors and generalized Bayes. In the large p (parameter dimension) and small n (sample size) setting, sparsity-inducing priors have been successfully used for linear regression when p ≫ n. However, their application to implicit generative models remains challenging due to the intractability of the likelihood function. To address this, we adapt a scoring rule posterior based on a strictly proper scoring rule, following the generalized Bayes framework. We then propose a block SGMCMC within Gibbs sampling mechanism. This enables efficient exploration of high-dimensional parameter spaces by identifying a sparse representation of neural implicit generative models. We demonstrate the performance of our proposed method through experiments on p ≫ n linear regression and an application of neural generative models.
ER -