Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection
Abstract
Automatic relevance determination (ARD), the default tool for variable selection in Gaussian-process (GP) regression, ranks inputs by inverse lengthscales -- which measure how fast a function varies, not how much an input contributes to prediction -- and offers no calibrated rule for deciding which inputs to keep. The prediction-centred alternative, the derivative sensitivity νj = 𝔼[(∂ f/∂ xj)2], is available in closed form from a fitted GP, but turning it into a selection rule is harder than it looks: at a null input the estimator is a degenerate quadratic form, so Wald and Bernstein-von Mises cutoffs are anti-conservative, and the natural residual bootstrap is mis-scaled. We show that a studentized multiplier bootstrap of the GP derivative process repairs both, prove its validity through an invariance principle for quadratic forms, and obtain asymptotic family-wise and false-discovery-rate control across inputs. Over 100 replications the rule controls FDR wherever inputs are truly null, while uncalibrated derivative rankings breach the target by up to 2x and a Bernstein-von Mises cutoff by 2.2x; at matched FDR it loses no power; it holds under a Matérn kernel and input correlation up to 0.99; on real data with planted and authentic null inputs it admits 5-12x fewer spurious inputs; it costs 5-18% of the GP fit; and a block-averaged variant retains validity at cost linear in n.
Related
- Sparsity by Default: The Theory and Practice of ARD in Gaussian Process Regression for Variable Selection
- R2-GP: Variance-Calibrated Priors for Additive Gaussian Processes
- Derivative-Informed Dimension Reduction of Stochastic Differential Equations: A Path-Space Sensitivity Approach
- Derivative Gaussian Processes on a Two-Direction Budget
- Calibrated Bayesian Selection of Schema Subgraphs for Database Question Answering
BibTeX
@misc{cai2026calibrated,
title = {{Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection}},
author = {Jia Cai},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.33549},
eprint = {2609.33549},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.33549},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection
AU - Jia Cai
PY - 2026
DA - 2026/09/27
PB - arXiv
DO - 10.48550/arxiv.2609.33549
UR - https://arxiv.org/abs/2609.33549
AB - Automatic relevance determination (ARD), the default tool for variable selection in Gaussian-process (GP) regression, ranks inputs by inverse lengthscales -- which measure how fast a function varies, not how much an input contributes to prediction -- and offers no calibrated rule for deciding which inputs to keep. The prediction-centred alternative, the derivative sensitivity ν_j = 𝔼[(∂ f/∂ x_j)^2], is available in closed form from a fitted GP, but turning it into a selection rule is harder than it looks: at a null input the estimator is a degenerate quadratic form, so Wald and Bernstein-von Mises cutoffs are anti-conservative, and the natural residual bootstrap is mis-scaled. We show that a studentized multiplier bootstrap of the GP derivative process repairs both, prove its validity through an invariance principle for quadratic forms, and obtain asymptotic family-wise and false-discovery-rate control across inputs. Over 100 replications the rule controls FDR wherever inputs are truly null, while uncalibrated derivative rankings breach the target by up to 2x and a Bernstein-von Mises cutoff by 2.2x; at matched FDR it loses no power; it holds under a Matérn kernel and input correlation up to 0.99; on real data with planted and authentic null inputs it admits 5-12x fewer spurious inputs; it costs 5-18% of the GP fit; and a block-averaged variant retains validity at cost linear in n.
ER -