Canonical Layer Inference for Deep Gaussian Processes via Symmetry-Breaking Augmentation
Abstract
Deep Gaussian processes (DGPs) are flexible priors for compositional regression functions, but inference for intermediate layers faces questions. The basic obstruction is an adjacent-layer symmetry: applying an invertible transformation and its inverse to two adjacent layers leaves the composed regression map unchanged. Under explicit symmetry and prior-comparability conditions, we show that the posterior may not concentrate on any prescribed intermediate-layer representation, even when the composed regression function is estimable. We then introduce an augmented ρ-fractional posterior that reweights layer decompositions by a non-saturating identity-deviation penalty. The augmentation changes posterior odds between symmetry-related decompositions by an exponential penalty ratio. Under proper choices of the penalty and a prior mass condition, it preserves function-level fractional posterior contraction, while shrinking the redundant layers towards the identity mapping. Thus, the likelihood does not identify a unique ``true'' set of layers; the augmentation favors identity-biased parsimonious decompositions within an equivalence class of layers; uniqueness of a selected representative is not established.
Related
- Local increment inference for time-inhomogeneous drift in Gaussian processes
- Neural Simulation-Based Inference for space-time Gaussian processes
- Adaptive multi-resolution Gaussian processes: Scalable exact inference with naturally data-sparse covariance matrices
- Penalized Likelihood Inference in Highly Multivariate Sparse Spectral Graphical Gaussian Processes
- An invariant variational proof for canonical processes with independent symmetric log-concave tails
BibTeX
@misc{zhu2026canonical,
title = {{Canonical Layer Inference for Deep Gaussian Processes via Symmetry-Breaking Augmentation}},
author = {Yichen Zhu and Hengrui Luo},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.23842},
eprint = {2610.23842},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.23842},
note = {CC BY-NC-SA 4.0},
}
RIS
TY - UNPB
TI - Canonical Layer Inference for Deep Gaussian Processes via Symmetry-Breaking Augmentation
AU - Yichen Zhu
AU - Hengrui Luo
PY - 2026
DA - 2026/10/16
PB - arXiv
DO - 10.48550/arxiv.2610.23842
UR - https://arxiv.org/abs/2610.23842
AB - Deep Gaussian processes (DGPs) are flexible priors for compositional regression functions, but inference for intermediate layers faces questions. The basic obstruction is an adjacent-layer symmetry: applying an invertible transformation and its inverse to two adjacent layers leaves the composed regression map unchanged. Under explicit symmetry and prior-comparability conditions, we show that the posterior may not concentrate on any prescribed intermediate-layer representation, even when the composed regression function is estimable. We then introduce an augmented ρ-fractional posterior that reweights layer decompositions by a non-saturating identity-deviation penalty. The augmentation changes posterior odds between symmetry-related decompositions by an exponential penalty ratio. Under proper choices of the penalty and a prior mass condition, it preserves function-level fractional posterior contraction, while shrinking the redundant layers towards the identity mapping. Thus, the likelihood does not identify a unique ``true'' set of layers; the augmentation favors identity-biased parsimonious decompositions within an equivalence class of layers; uniqueness of a selected representative is not established.
ER -