Derivative Gaussian Processes on a Two-Direction Budget

Abstract

Gradient observations promise more accurate Gaussian process (GP) surrogates, but the cost of incorporating them has long stood in the way of realizing that promise. We propose a derivative GP with a budget of just two directions per observed gradient. One direction focuses on each gradient's direct contribution to target prediction, while the other aggregates its indirect contributions through correlations with the conditioning function values. Within a Vecchia approximation, where each prediction conditions on m nearby inputs in d dimensions, this construction represents their md gradient coordinates using at most 2m directional derivatives, giving 𝒪(m³) dense factorization cost per prediction target. For general conditioning sets, we bound the posterior approximation error relative to using full gradients and characterize when the error is small or the approximation is exact. In simulations, our method matches the accuracy of a leading exact gradient-reduction method at equal conditioning set size. Because its cost grows much more slowly with that size, it can use conditioning sets well beyond the memory limit of the exact method, reaching lower prediction error with a small fraction of the time and memory. Notably, our method can exploit gradient observations while requiring less computation time or memory than function-only GP baselines.

A Feasible Direction Method with Active-Set Identification for Derivative-Free Optimization
Jeffrey Larson, Matt Menickelly, Lara Zebiane
arXiv28 Oct 2026PreprintarXiv licenceFull textOptimisation & controlarXiv
The AECM Algorithm for Deterministic Maximum Likelihood Direction Finding in the Presence of Gaussian Mixture Noise
Mingyan Gong, Bin Lyu
arXiv17 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv
Learning Non-Gaussian SEMs with Conditional Risk, and Why the Direction of Recursion Matters
Ujas Shah, Panagiota Birmpa, Rebecca Morrison
arXiv9 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv
Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection
Jia Cai
arXiv27 Sep 2026PreprintCC BY-NC-ND 4.0Full textNumerical analysis & scientific computing StatisticsarXiv
Quasihelix properties of selected Volterra Gaussian processes
Yuliya Mishura, Kostiantyn Ralchenko
arXiv26 Oct 2026PreprintarXiv licenceFull textProbabilityarXiv

Cite

BibTeX RIS

BibTeX

@misc{seung2026derivative,
  title = {{Derivative Gaussian Processes on a Two-Direction Budget}},
  author = {Hyunseok Seung and Matthias Katzfuss},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.10428},
  eprint = {2610.10428},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.10428},
}

RIS

TY  - UNPB
TI  - Derivative Gaussian Processes on a Two-Direction Budget
AU  - Hyunseok Seung
AU  - Matthias Katzfuss
PY  - 2026
DA  - 2026/10/07
PB  - arXiv
DO  - 10.48550/arxiv.2610.10428
UR  - https://arxiv.org/abs/2610.10428
AB  - Gradient observations promise more accurate Gaussian process (GP) surrogates, but the cost of incorporating them has long stood in the way of realizing that promise. We propose a derivative GP with a budget of just two directions per observed gradient. One direction focuses on each gradient's direct contribution to target prediction, while the other aggregates its indirect contributions through correlations with the conditioning function values. Within a Vecchia approximation, where each prediction conditions on m nearby inputs in d dimensions, this construction represents their md gradient coordinates using at most 2m directional derivatives, giving 𝒪(m^3) dense factorization cost per prediction target. For general conditioning sets, we bound the posterior approximation error relative to using full gradients and characterize when the error is small or the approximation is exact. In simulations, our method matches the accuracy of a leading exact gradient-reduction method at equal conditioning set size. Because its cost grows much more slowly with that size, it can use conditioning sets well beyond the memory limit of the exact method, reaching lower prediction error with a small fraction of the time and memory. Notably, our method can exploit gradient observations while requiring less computation time or memory than function-only GP baselines.
ER  -

Abstract

Related

Cite