SPARK: A General Goodness-of-Fit Assessment via Residual Projection
Abstract
Goodness-of-fit testing is a basic tool for assessing whether a fitted procedure has captured the systematic information contained in the covariates. While traditional theory has largely focused on parametric regression models, modern data analysis increasingly relies on flexible black-box learners, whose predictive success alone is insufficient to assess model accuracy. In this paper, we propose SPARK, a general framework for goodness-of-fit testing that applies to traditional statistical models and general black-box learning procedures, continuous and binary responses, and low- and high-dimensional predictors. Based on a debiasing strategy, the residuals from an initial fit of a learning procedure are projected onto nearly orthogonal directions to extract any remaining signal. To capture information across all projection directions, we propose a kernel-based projection method and establish both its asymptotic properties and the consistency of a bootstrap procedure. Comprehensive simulations and real data analyses illustrate the effectiveness and flexibility of our proposed method.
Related
- Goodness-of-fit testing for the Pareto type-I distribution based on a mean residual life characterization
- KKT Residual Sliding Manifolds: A Projection-Free Structured Finite-Time Solver for Nonlinear Constrained Optimisation
- Mordukhovich Derivatives and Covering Constant for Set-Valued Metric Projection in General Banach Spaces
- Residual Feedback for Transformed-State Equilibrium Seeking via General Variational Inequalities
- Algorithmic threshold for high-dimensional projection pursuit I: general theory
BibTeX
@misc{liu2026spark,
title = {{SPARK: A General Goodness-of-Fit Assessment via Residual Projection}},
author = {Xingwei Liu and Yuhong Yang and Wangli Xu},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.37705},
eprint = {2609.37705},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.37705},
}
RIS
TY - UNPB
TI - SPARK: A General Goodness-of-Fit Assessment via Residual Projection
AU - Xingwei Liu
AU - Yuhong Yang
AU - Wangli Xu
PY - 2026
DA - 2026/09/29
PB - arXiv
DO - 10.48550/arxiv.2609.37705
UR - https://arxiv.org/abs/2609.37705
AB - Goodness-of-fit testing is a basic tool for assessing whether a fitted procedure has captured the systematic information contained in the covariates. While traditional theory has largely focused on parametric regression models, modern data analysis increasingly relies on flexible black-box learners, whose predictive success alone is insufficient to assess model accuracy. In this paper, we propose SPARK, a general framework for goodness-of-fit testing that applies to traditional statistical models and general black-box learning procedures, continuous and binary responses, and low- and high-dimensional predictors. Based on a debiasing strategy, the residuals from an initial fit of a learning procedure are projected onto nearly orthogonal directions to extract any remaining signal. To capture information across all projection directions, we propose a kernel-based projection method and establish both its asymptotic properties and the consistency of a bootstrap procedure. Comprehensive simulations and real data analyses illustrate the effectiveness and flexibility of our proposed method.
ER -