Edge-preserving Bayesian neural networks via hierarchical sparsity-promoting priors

Abstract

Recovering piecewise-smooth functions with jump discontinuities from noisy, indirect, and incomplete data is a central task in computational inverse problems. Bayesian neural networks (BNNs) with heavy-tailed weight priors have emerged as effective edge-preserving priors for such problems. However, posterior inference for these models remains computationally challenging, and existing approaches settle for point estimates obtained by directly optimizing heavy-tailed log-densities. We propose GSBL-BNNs, which model the neural network parameters hierarchically using sparsity-promoting priors from generalized sparse Bayesian learning (GSBL). The hierarchical formulation recovers several popular marginal priors, including Laplace and Student's t (including Cauchy). A key modeling ingredient, novel among edge-preserving BNNs and native to the hierarchical formulation, couples each neuron's bias to its weights through a shared hyperparameter, so that the induced jump discontinuities are free to occur anywhere in the domain while the conditionally Gaussian structure underlying efficient inference is preserved. Furthermore, we develop a block-coordinate descent (BCD) algorithm for maximum a posteriori (MAP) estimation. This BCD approach alternates between standard network training with iteratively reweighted ℓ²-regularization and exact, closed-form hyperparameter updates for gamma and inverse gamma hyperpriors. Building on the BCD algorithm, we perform approximate Bayesian inference and uncertainty quantification (UQ) using deep ensembles. Computational experiments---including signal denoising, deblurring, and inverting a two-dimensional hyperbolic conservation law--- show that GSBL-BNNs deliver accurate, edge-preserving reconstructions with built-in UQ. Compared with Gaussian- and Cauchy-BNNs, they yield more accurate predictive means and avoid overconfident uncertainty estimates.

Sparsity-preserving means on a homogeneous convex cone
Khalid Koufany
HAL4 Oct 2026PreprintarXiv licenceFull textAnalysis & PDEs Geometry & topologyHALarXiv
Bayesian Nonparametric Factor Analysis via Marginalized Dirichlet Process Column Clustering with Spike-and-Slab Sparsity
Durba Bhattacharya, Sourabh Bhattacharya
arXiv28 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv
Linear-Time Bilevel Clipping operators for Structured Sparsity in Neural Networks
Michel Barlaud, Jean-Paul Marmorat, Guillaume Perez
HAL4 Aug 2026PreprintCC BY-NC 4.0Full textStatisticsHAL
Sparsity-Corrected Tensor Clustering for Multi-Layer Networks
Xuefei Wang, Gaorong Li, Wenyang Zhang
arXiv16 Oct 2026PreprintarXiv licenceFull textStatisticsarXiv
Explaining the Saliency Map Sparsity of Adversarially-Trained Neural Networks
Yannick Lunk, Atell Yehor Krasnopolsky, Damien Garreau +1
arXiv7 Oct 2026PreprintarXiv licenceFull textOptimisation & control StatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{habscheid2026edgepreserving,
  title = {{Edge-preserving Bayesian neural networks via hierarchical sparsity-promoting priors}},
  author = {Jan Habscheid and Jan Glaubitz},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.17023},
  eprint = {2610.17023},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.17023},
}

RIS

TY  - UNPB
TI  - Edge-preserving Bayesian neural networks via hierarchical sparsity-promoting priors
AU  - Jan Habscheid
AU  - Jan Glaubitz
PY  - 2026
DA  - 2026/10/12
PB  - arXiv
DO  - 10.48550/arxiv.2610.17023
UR  - https://arxiv.org/abs/2610.17023
AB  - Recovering piecewise-smooth functions with jump discontinuities from noisy, indirect, and incomplete data is a central task in computational inverse problems. Bayesian neural networks (BNNs) with heavy-tailed weight priors have emerged as effective edge-preserving priors for such problems. However, posterior inference for these models remains computationally challenging, and existing approaches settle for point estimates obtained by directly optimizing heavy-tailed log-densities. We propose GSBL-BNNs, which model the neural network parameters hierarchically using sparsity-promoting priors from generalized sparse Bayesian learning (GSBL). The hierarchical formulation recovers several popular marginal priors, including Laplace and Student's t (including Cauchy). A key modeling ingredient, novel among edge-preserving BNNs and native to the hierarchical formulation, couples each neuron's bias to its weights through a shared hyperparameter, so that the induced jump discontinuities are free to occur anywhere in the domain while the conditionally Gaussian structure underlying efficient inference is preserved. Furthermore, we develop a block-coordinate descent (BCD) algorithm for maximum a posteriori (MAP) estimation. This BCD approach alternates between standard network training with iteratively reweighted ℓ^2-regularization and exact, closed-form hyperparameter updates for gamma and inverse gamma hyperpriors. Building on the BCD algorithm, we perform approximate Bayesian inference and uncertainty quantification (UQ) using deep ensembles. Computational experiments---including signal denoising, deblurring, and inverting a two-dimensional hyperbolic conservation law--- show that GSBL-BNNs deliver accurate, edge-preserving reconstructions with built-in UQ. Compared with Gaussian- and Cauchy-BNNs, they yield more accurate predictive means and avoid overconfident uncertainty estimates.
ER  -

Abstract

Related

Cite