Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
Abstract
Conditional density estimation targets the full distribution of a response given covariates, as required, for example, for per-galaxy photometric redshifts. We develop a scalable Bayesian estimator based on the logistic Gaussian process. The log conditional density has a separable covariance: a Matérn kernel along the response, represented in a truncated Fourier basis on a circle, and a covariate kernel represented by Nyström features, which accommodate non-stationary kernels with input-dependent amplitudes and length scales. Instead of a Laplace or variational approximation, we sample the latent field of this finite-feature model. Given the hyperparameters, its posterior is strongly log-concave with a uniformly bounded Hessian, and we draw from it by simulating kinetic Langevin dynamics with symmetric minibatch splitting in Kronecker-whitened coordinates. Marginal-likelihood gradients follow from Fisher's identity as posterior expectations. Under the conditions of our analysis their bias is controlled by the sampler's step size and run length, and the predictive averages over the non-Gaussian latent posterior instead of a Gaussian around its mode. On photometric-redshift benchmarks with up to 3.9 million training observations, trained on a single GPU, the estimator is competitive with state-of-the-art tabular foundation models on density and calibration metrics.
Related
- Logistic Gaussian process density regression: a generalized Bayesian approach
- Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
- Mapping urban heat from citizen-collected data: a scalable Gaussian process approach to correcting for preferential sampling
- Data generation for the logistic regression model under controlled conditions
- A penalized logistic generalized regression estimator
BibTeX
@misc{paulin2026scalable,
title = {{Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling}},
author = {Daniel Paulin and Ádám Jung and András A. Benczúr},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.09591},
eprint = {2610.09591},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.09591},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling
AU - Daniel Paulin
AU - Ádám Jung
AU - András A. Benczúr
PY - 2026
DA - 2026/10/07
PB - arXiv
DO - 10.48550/arxiv.2610.09591
UR - https://arxiv.org/abs/2610.09591
AB - Conditional density estimation targets the full distribution of a response given covariates, as required, for example, for per-galaxy photometric redshifts. We develop a scalable Bayesian estimator based on the logistic Gaussian process. The log conditional density has a separable covariance: a Matérn kernel along the response, represented in a truncated Fourier basis on a circle, and a covariate kernel represented by Nyström features, which accommodate non-stationary kernels with input-dependent amplitudes and length scales. Instead of a Laplace or variational approximation, we sample the latent field of this finite-feature model. Given the hyperparameters, its posterior is strongly log-concave with a uniformly bounded Hessian, and we draw from it by simulating kinetic Langevin dynamics with symmetric minibatch splitting in Kronecker-whitened coordinates. Marginal-likelihood gradients follow from Fisher's identity as posterior expectations. Under the conditions of our analysis their bias is controlled by the sampler's step size and run length, and the predictive averages over the non-Gaussian latent posterior instead of a Gaussian around its mode. On photometric-redshift benchmarks with up to 3.9 million training observations, trained on a single GPU, the estimator is competitive with state-of-the-art tabular foundation models on density and calibration metrics.
ER -