Differentiating Minimal-Norm Solutions to Parametric Optimization Problems
Abstract
Differentiating through parametric optimization problems is central to bilevel programming and meta-learning, often accomplished using approximate implicit differentiation. The implicit function theorem requires inverting a partial Jacobian of the optimality condition, which fails when there are many solutions. Nonetheless, in such cases it is possible to relax invertibility to a strictly weaker uniform range condition, under which it is shown that the minimal-norm solution mapping admits generalized derivatives by using a limiting Tikhonov regularization argument and conservative set-valued field theory. With additional control on the eigenvalues of the generalized Hessians, a pseudoinverse formula is justified. This is established for a class of smooth convex objectives and extended to nonsmooth composite problems. These assumptions are verified for Least-Squares, Huber regression and LASSO. The resulting extension of nonsmooth implicit differentiation to ill-posed settings is examined experimentally on data poisoning and data hypercleaning problems.
Related
- Parametric Families of Minimal Positive Cubature Formulas of Degrees 4 and 6 on the 2-Sphere
- dOPT: Differentiating Conic Optimization via Geometric Reduction
- Existence of three solutions for parametric p-Laplacian equations in ℝN
- Analytic Solutions To Fully Quantized Parametric Amplification
- Variationally correct loss functions for nonlinear parametric problems and their neural approximations
BibTeX
@misc{plaqueventjourdain2026differentiating,
title = {{Differentiating Minimal-Norm Solutions to Parametric Optimization Problems}},
author = {Baptiste Plaquevent-Jourdain and Jalal Fadili and Antonio Silveti-Falls},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2608.28899},
eprint = {2608.28899},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2608.28899},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Differentiating Minimal-Norm Solutions to Parametric Optimization Problems
AU - Baptiste Plaquevent-Jourdain
AU - Jalal Fadili
AU - Antonio Silveti-Falls
PY - 2026
DA - 2026/08/28
PB - arXiv
DO - 10.48550/arxiv.2608.28899
UR - https://arxiv.org/abs/2608.28899
AB - Differentiating through parametric optimization problems is central to bilevel programming and meta-learning, often accomplished using approximate implicit differentiation. The implicit function theorem requires inverting a partial Jacobian of the optimality condition, which fails when there are many solutions. Nonetheless, in such cases it is possible to relax invertibility to a strictly weaker uniform range condition, under which it is shown that the minimal-norm solution mapping admits generalized derivatives by using a limiting Tikhonov regularization argument and conservative set-valued field theory. With additional control on the eigenvalues of the generalized Hessians, a pseudoinverse formula is justified. This is established for a class of smooth convex objectives and extended to nonsmooth composite problems. These assumptions are verified for Least-Squares, Huber regression and LASSO. The resulting extension of nonsmooth implicit differentiation to ill-posed settings is examined experimentally on data poisoning and data hypercleaning problems.
ER -