Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample
Abstract
We estimate the conditional population-risk curve of a realized smooth nonconvex gradient flow from the training sample. Flow approximate leave-one-out (Flow-ALO) propagates a deletion response and evaluates omitted observations at approximate deleted paths. The risk-curve error decomposes into response approximation, exact-LOO fluctuation, and deletion-to-full risk transfer. On each fixed finite horizon, bounded centered training-loss gradients, a one-sided Hessian lower bound, locally Lipschitz Hessians, and a strict tube-closure condition yield an explicit (n−1)−2 bound for the deletion-response error. Bounded evaluation-loss gradients transfer the deletion-response bound to the score without requiring the Hessian to be invertible. Direct first-order jackknife cancellation and exact-LOO concentration control deletion-to-full risk transfer and fluctuation, respectively, completing recovery of the conditional population-risk curve. For bounded smooth two-layer mean-field networks training both layers, the score-error bound is uniform in width.
Related
- Preconditioned Three-Term Conjugate Gradient Framework for Nonconvex Finite Minimax Problems
- Asymptotic Analysis of Gradient Mapping-type Stationarity Measure for the Sum of Nonconvex Nonsmooth Functions and Applications to Proximal Gradient-type Algorithms
- Stochastic Gradient Descent Ascent is Suboptimal for Nonconvex-PŁ Min-Max Games
- A Proximal Gradient Method for Nonconvex Nonsmooth Optimization with Applications
- The Role of Gradient Modification in Heavy-Tailed Nonconvex Stochastic Min-Max Optimization
BibTeX
@misc{song2026estimating,
title = {{Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample}},
author = {Mingzhi Song},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.30261},
eprint = {2608.30261},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.30261},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Estimating Population-Risk Curves Along Nonconvex Gradient Flows from the Training Sample
AU - Mingzhi Song
PY - 2026
DA - 2026/08/31
PB - arXiv
DO - 10.48550/arxiv.2608.30261
UR - https://arxiv.org/abs/2608.30261
AB - We estimate the conditional population-risk curve of a realized smooth nonconvex gradient flow from the training sample. Flow approximate leave-one-out (Flow-ALO) propagates a deletion response and evaluates omitted observations at approximate deleted paths. The risk-curve error decomposes into response approximation, exact-LOO fluctuation, and deletion-to-full risk transfer. On each fixed finite horizon, bounded centered training-loss gradients, a one-sided Hessian lower bound, locally Lipschitz Hessians, and a strict tube-closure condition yield an explicit (n−1)^−2 bound for the deletion-response error. Bounded evaluation-loss gradients transfer the deletion-response bound to the score without requiring the Hessian to be invertible. Direct first-order jackknife cancellation and exact-LOO concentration control deletion-to-full risk transfer and fluctuation, respectively, completing recovery of the conditional population-risk curve. For bounded smooth two-layer mean-field networks training both layers, the score-error bound is uniform in width.
ER -