Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation

Abstract

Deep Gaussian processes (DGPs) are flexible surrogate models for computer experiments with nonstationary response surfaces, but inference over their latent representations remains computationally challenging. Existing methods typically rely on variational inference or Markov chain Monte Carlo (MCMC), which impose variational-family assumptions or require costly posterior sampling. We propose the latent maximum-a-posteriori deep Gaussian process (LaMP-DGP), which uses gradient-based optimization to obtain MAP estimates of the latent representations and enables efficient probabilistic prediction through linked-GP calculations. To scale LaMP-DGP to large datasets, we develop a mini-batch optimization algorithm and, under a polynomial mini-batch bias-decay condition, establish a stationarity bound for latent-state optimization that motivates a growing batch-size schedule. Across numerical experiments and real-data analyses, LaMP-DGP provides consistently competitive predictive performance relative to doubly stochastic variational inference, stochastic gradient Hamiltonian Monte Carlo, and fully Bayesian Vecchia DGP inference. Computationally, LaMP-DGP is substantially faster than the Vecchia MCMC approach while remaining broadly competitive with the stochastic-gradient alternatives. These results position LaMP-DGP as a competitive and computationally efficient alternative for DGP inference. An R package, "LaMPDGP", is available for implementing the proposed method.

Causal Mediation Analysis with High-Dimensional Surrogates: Identification, Estimation and Inference under Latent Factor Models
Himani Yadav, Shi Bo, AmirEmad Ghassami +1
arXiv2 Nov 2026PreprintCC BY 4.0Full textStatisticsarXiv
Amortized Structured Stochastic Variational Inference for Gaussian Process Latent Variable Models
Maksym Tretiakov, Sarah Filippi, Vincent Fortuin +3
arXiv9 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv
Efficient estimation methods for non-Gaussian regression models in continuous time
Evgeny A Pchelintsev, Sergey Pergamenshchikov, Maria Povzun
Annals of the Institute of Statistical Mathematics7 Sep 2026Peer-reviewedPublisher termsFull text5 citedStatisticsHAL
Efficient Gaussian Process Modeling for Large Replicated and Censored Data in Industrial Research
Ying Zhang, Emily L. Kang, Shuo Li
arXiv3 Nov 2026PreprintarXiv licenceFull textStatisticsarXiv
Practical Algebraic Parameter Estimation for Noisy Data via Gaussian Process Regression
Oren Bassik, Alexander Demin, Alexey Ovchinnikov
arXiv24 Sep 2026PreprintCC BY 4.0Full textOptimisation & controlarXiv

Cite

BibTeX RIS

BibTeX

@misc{chang2026efficient,
  title = {{Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation}},
  author = {Chun-Yi Chang and Chih-Li Sung},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.38038},
  eprint = {2610.38038},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.38038},
  note = {CC BY 4.0},
}

RIS

TY  - UNPB
TI  - Efficient Deep Gaussian Process Surrogates via Mini-Batch Latent MAP Estimation
AU  - Chun-Yi Chang
AU  - Chih-Li Sung
PY  - 2026
DA  - 2026/10/27
PB  - arXiv
DO  - 10.48550/arxiv.2610.38038
UR  - https://arxiv.org/abs/2610.38038
AB  - Deep Gaussian processes (DGPs) are flexible surrogate models for computer experiments with nonstationary response surfaces, but inference over their latent representations remains computationally challenging. Existing methods typically rely on variational inference or Markov chain Monte Carlo (MCMC), which impose variational-family assumptions or require costly posterior sampling. We propose the latent maximum-a-posteriori deep Gaussian process (LaMP-DGP), which uses gradient-based optimization to obtain MAP estimates of the latent representations and enables efficient probabilistic prediction through linked-GP calculations. To scale LaMP-DGP to large datasets, we develop a mini-batch optimization algorithm and, under a polynomial mini-batch bias-decay condition, establish a stationarity bound for latent-state optimization that motivates a growing batch-size schedule. Across numerical experiments and real-data analyses, LaMP-DGP provides consistently competitive predictive performance relative to doubly stochastic variational inference, stochastic gradient Hamiltonian Monte Carlo, and fully Bayesian Vecchia DGP inference. Computationally, LaMP-DGP is substantially faster than the Vecchia MCMC approach while remaining broadly competitive with the stochastic-gradient alternatives. These results position LaMP-DGP as a competitive and computationally efficient alternative for DGP inference. An R package, "LaMPDGP", is available for implementing the proposed method.
ER  -

Abstract

Related

Cite