Logistic Gaussian process density regression: a generalized Bayesian approach
Abstract
Density regression extends conventional parametric regression by allowing the entire distribution of the response to vary flexibly with covariates rather than just low-order moments. In the Bayesian setting, logistic Gaussian process (GP) priors have been widely used for density estimation and extend naturally to density regression. The prior can be centred on a base density model, with the nonparametric component providing an interpretable correction that is useful for model criticism. However, logistic GP density regression models have seen limited use, since they require computation of a normalizing constant for every observation, typically via numerical integration. We address this difficulty by proposing a generalized Bayesian approach using a loss function based on the Hyvärinen score. The Hyvärinen score depends only on derivatives of the log density with respect to the response, eliminating the need to compute normalizing constants. Since GP computations remain expensive, we also employ sparse inducing point approximations and variational inference to develop a scalable approach. We demonstrate the method on one simulated and two real datasets, including a German weather dataset with more than 150,000 observations.
Related
- Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
- Using Bayesian Sequential Logistic Regression to Model the Organ Transplant Selection Process
- A penalized logistic generalized regression estimator
- Variational Bayesian inference for logistic regression models for group testing data: Comparing a new hybrid variational inference approach to traditional methods.
- Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
BibTeX
@misc{chen2026logistic,
title = {{Logistic Gaussian process density regression: a generalized Bayesian approach}},
author = {Zichuan Chen and Lucas Kock and Jeong Eun Lee and David J. Nott},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.22915},
eprint = {2606.22915},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.22915},
note = {CC BY-SA 4.0},
}
RIS
TY - UNPB
TI - Logistic Gaussian process density regression: a generalized Bayesian approach
AU - Zichuan Chen
AU - Lucas Kock
AU - Jeong Eun Lee
AU - David J. Nott
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2606.22915
UR - https://arxiv.org/abs/2606.22915
AB - Density regression extends conventional parametric regression by allowing the entire distribution of the response to vary flexibly with covariates rather than just low-order moments. In the Bayesian setting, logistic Gaussian process (GP) priors have been widely used for density estimation and extend naturally to density regression. The prior can be centred on a base density model, with the nonparametric component providing an interpretable correction that is useful for model criticism. However, logistic GP density regression models have seen limited use, since they require computation of a normalizing constant for every observation, typically via numerical integration. We address this difficulty by proposing a generalized Bayesian approach using a loss function based on the Hyvärinen score. The Hyvärinen score depends only on derivatives of the log density with respect to the response, eliminating the need to compute normalizing constants. Since GP computations remain expensive, we also employ sparse inducing point approximations and variational inference to develop a scalable approach. We demonstrate the method on one simulated and two real datasets, including a German weather dataset with more than 150,000 observations.
ER -