NPBoost: Neural Processes with Gradient-Boosted Fixed Effects
Abstract
Neural Processes (NPs) are model-based meta-learners that implicitly learn a stochastic process and adapt to a new task from a small context set. Most extensions of NPs focus on improving the neural network architecture. We instead develop an extension motivated by the shared hierarchical interpretation of meta-learning and mixed-effects models. Specifically, we introduce Neural Process Boosting (NPBoost), which decomposes structured response variability into tree-boosted fixed effects shared across tasks and NP random effects that capture stochastic task-to-task variation. We propose to train the two components jointly using a boosting algorithm in which an NP learns residual task-specific structure and a tree ensemble estimates common patterns across tasks. Across synthetic and real-world tabular meta-learning problems, this decomposition improves over a standard NP when the shared structure contains discontinuities or other irregular patterns that boosted trees can represent effectively.
Related
- Conditional Weight of Evidence for Gradient-Boosted Credit Models: Estimation, Additivity and Its Relation to Adverse Action Reasons
- Geotechnically Informed Monitoring-Data Fusion and Gradient-Boosted Regression for Instrumented Driven Piles
- Boosted Physics-Informed Neural Networks for Solving Homogeneous and Non-Homogeneous Lane-Emden Type Equations
- Global optimization by random perturbation of the gradient method with a fixed parameter
- Non-equilibrium fluctuations of gradient exclusion processes in dimension d≤ 3
BibTeX
@misc{nava2026npboost,
title = {{NPBoost: Neural Processes with Gradient-Boosted Fixed Effects}},
author = {Andrea Nava and Ken Rölli and Armin Begic and Fabio Sigrist},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.28122},
eprint = {2609.28122},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.28122},
}
RIS
TY - UNPB
TI - NPBoost: Neural Processes with Gradient-Boosted Fixed Effects
AU - Andrea Nava
AU - Ken Rölli
AU - Armin Begic
AU - Fabio Sigrist
PY - 2026
DA - 2026/09/23
PB - arXiv
DO - 10.48550/arxiv.2609.28122
UR - https://arxiv.org/abs/2609.28122
AB - Neural Processes (NPs) are model-based meta-learners that implicitly learn a stochastic process and adapt to a new task from a small context set. Most extensions of NPs focus on improving the neural network architecture. We instead develop an extension motivated by the shared hierarchical interpretation of meta-learning and mixed-effects models. Specifically, we introduce Neural Process Boosting (NPBoost), which decomposes structured response variability into tree-boosted fixed effects shared across tasks and NP random effects that capture stochastic task-to-task variation. We propose to train the two components jointly using a boosting algorithm in which an NP learns residual task-specific structure and a tree ensemble estimates common patterns across tasks. Across synthetic and real-world tabular meta-learning problems, this decomposition improves over a standard NP when the shared structure contains discontinuities or other irregular patterns that boosted trees can represent effectively.
ER -