On Basis Function Selection for Sparse Gaussian Process Regression
Abstract
Sparse Gaussian processes achieve O(N) inference by replacing the kernel with an appropriate expansion in a fixed basis {ϕj} on the input space. Given a compute budget M ≪ N, practitioners conventionally truncate the basis to its first M entries. Nothing in the formalism, however, prevents one from selecting only those M basis functions that matter for the data at hand. This would avoid spending budget on basis functions where there is no signal, but it requires a criterion for ranking the candidates. We propose three such criteria derived from an information-theoretic view of the basis-function selection problem. Each criterion matches a different state of knowledge at selection time: a no-data state, a no-prior state, and an in-between state. We then study the performance of truncation versus selection strategies on six UCI regression benchmarks across three basis families: Hilbert-space Gaussian processes (HSGP), variational Fourier features (VFF), and variational inducing spherical harmonics (VISH). We observe that the no-data criterion is a safe default, matching or improving on truncation for HSGP, VFF and VISH, with substantial gains for VISH and improvements over a recently developed selection heuristic for that basis family. The data-aware no-prior and in-between criteria provide substantial gains over truncation specifically for HSGP, which is the most broadly used of the three families in practice.
Related
- Sequential sparse Gaussian process quantile regression
- Sparsity by Default: The Theory and Practice of ARD in Gaussian Process Regression for Variable Selection
- A Gaussian Process Regression-Enhanced Two-Sweep Parameterized Power Method for Computing Extreme Eigenpairs
- Bayes Factor Testing of Linear and Nonlinear Effects in Additive Gaussian Process Regression
- Positive, monotone, and convex Gaussian process regression with Laplacian eigenfunctions
BibTeX
@misc{soom2026basis,
title = {{On Basis Function Selection for Sparse Gaussian Process Regression}},
author = {Marnix Van Soom and Ivan De Boi},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.26624},
eprint = {2609.26624},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.26624},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - On Basis Function Selection for Sparse Gaussian Process Regression
AU - Marnix Van Soom
AU - Ivan De Boi
PY - 2026
DA - 2026/09/23
PB - arXiv
DO - 10.48550/arxiv.2609.26624
UR - https://arxiv.org/abs/2609.26624
AB - Sparse Gaussian processes achieve O(N) inference by replacing the kernel with an appropriate expansion in a fixed basis {ϕ_j} on the input space. Given a compute budget M ≪ N, practitioners conventionally truncate the basis to its first M entries. Nothing in the formalism, however, prevents one from selecting only those M basis functions that matter for the data at hand. This would avoid spending budget on basis functions where there is no signal, but it requires a criterion for ranking the candidates. We propose three such criteria derived from an information-theoretic view of the basis-function selection problem. Each criterion matches a different state of knowledge at selection time: a no-data state, a no-prior state, and an in-between state. We then study the performance of truncation versus selection strategies on six UCI regression benchmarks across three basis families: Hilbert-space Gaussian processes (HSGP), variational Fourier features (VFF), and variational inducing spherical harmonics (VISH). We observe that the no-data criterion is a safe default, matching or improving on truncation for HSGP, VFF and VISH, with substantial gains for VISH and improvements over a recently developed selection heuristic for that basis family. The data-aware no-prior and in-between criteria provide substantial gains over truncation specifically for HSGP, which is the most broadly used of the three families in practice.
ER -