Learn-Then-Differentiate Gradient Estimation
Abstract
Learn-then-differentiate (LTD) estimates gradients by fitting a model to simulation outputs and differentiating it. We develop a unified framework explaining what LTD differentiates and how accurately it estimates gradients. For models with a weighted representation, LTD differentiates a learned representation of the underlying probability measure. We then show how accuracy guarantees for fitted models translate into guarantees for gradients and higher-order derivatives, with rates approaching the standard Monte Carlo rate under suitable smoothness conditions. The framework recovers established results for kernel regression, local polynomial regression, and kernel ridge regression, and yields further guarantees for multiple kernel learning and smooth neural networks. These results provide a common foundation for understanding and analyzing LTD across learning methods.
Related
- Wait to Learn: Timing the Gradient Return in Over-the-Air Split Learning over an Aging Single-Antenna Channel
- Is √d Separation Necessary for Gradient EM to Learn Gaussian Mixtures in High Dimensions?
- Keep Pace to Learn: Movable Antennas Restore the Backpropagated Gradient Under User Mobility
- Spherical-Design Oracles for Zeroth-Order Optimisation: Deterministic, Variance-Free Gradient Estimation
- Online Covariance Estimation for Averaged Stochastic Gradient Descent on Riemannian Manifolds
BibTeX
@misc{lin2026learnthendifferentiate,
title = {{Learn-Then-Differentiate Gradient Estimation}},
author = {Nifei Lin and Qingkai Zhang and L. Jeff Hong},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.38842},
eprint = {2609.38842},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.38842},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Learn-Then-Differentiate Gradient Estimation
AU - Nifei Lin
AU - Qingkai Zhang
AU - L. Jeff Hong
PY - 2026
DA - 2026/09/30
PB - arXiv
DO - 10.48550/arxiv.2609.38842
UR - https://arxiv.org/abs/2609.38842
AB - Learn-then-differentiate (LTD) estimates gradients by fitting a model to simulation outputs and differentiating it. We develop a unified framework explaining what LTD differentiates and how accurately it estimates gradients. For models with a weighted representation, LTD differentiates a learned representation of the underlying probability measure. We then show how accuracy guarantees for fitted models translate into guarantees for gradients and higher-order derivatives, with rates approaching the standard Monte Carlo rate under suitable smoothness conditions. The framework recovers established results for kernel regression, local polynomial regression, and kernel ridge regression, and yields further guarantees for multiple kernel learning and smooth neural networks. These results provide a common foundation for understanding and analyzing LTD across learning methods.
ER -