When does least-squares residual minimization solve a differential equation?
Abstract
Least-squares residual minimization approximates a differential equation through an optimization problem. This paper investigates the conditions required to transition from stationary points and minimizing sequences of the residual functional to the exact solution. Starting from the classical orthogonality relation between the residual and the range of the linearized operator, we study how boundary conditions and parameterized variations determine what can be inferred about the residual at a stationary point. Smooth spurious critical points can occur even for well-posed problems; for linear equations, a compatibility condition yields an exact projection characterization. For a scalar conservation law with transonic boundary data, every minimizing sequence converges to a continuous function that does not solve the equation, even when a stationary entropy shock exists. We give conditions under which approaching this limit forces the parameter norms to diverge. A Hamilton--Jacobi example shows that a function can have zero residual without being the viscosity solution. Favorable results for a viscous model and uniformly convex Monge--Ampère critical points identify conditions that restore the corresponding implications. For piecewise-smooth trial families, the regularity needed to define the residual functional does not by itself justify the sampled gradient used in training. Moving residual jumps can contribute terms absent from that gradient; an explicit example has identically vanishing sampled gradients and a nonzero continuous derivative. Together, these results clarify what stationarity and sampled gradients imply about the residual, and under which conditions minimizing sequences lead to the intended solution.
Related
- A Large-Dimensional Analysis of Least-Squares Federated Learning via Generalized Total Variation Minimization
- FOSLS-deRhaNN: native de Rham neural classes for H(div) and H(curl) with applications to first-order system least-squares neural network methods for partial differential equations
- A COVID-19 transmission model with optimal control integrating BERT with LHS and least-squares fitting for real-world consistency
- A comparison of artificial neural networks and partial least squares modelling for the rapid detection of the microbial spoilage of beef fillets based on Fourier transform infrared spectral fingerprints
- A least-squares lower bound for capacity
BibTeX
@misc{esteveyague2026when,
title = {{When does least-squares residual minimization solve a differential equation?}},
author = {Carlos Esteve-Yagüe and Richard Tsai and Stanley Osher},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.35716},
eprint = {2609.35716},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.35716},
}
RIS
TY - UNPB
TI - When does least-squares residual minimization solve a differential equation?
AU - Carlos Esteve-Yagüe
AU - Richard Tsai
AU - Stanley Osher
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2609.35716
UR - https://arxiv.org/abs/2609.35716
AB - Least-squares residual minimization approximates a differential equation through an optimization problem. This paper investigates the conditions required to transition from stationary points and minimizing sequences of the residual functional to the exact solution. Starting from the classical orthogonality relation between the residual and the range of the linearized operator, we study how boundary conditions and parameterized variations determine what can be inferred about the residual at a stationary point. Smooth spurious critical points can occur even for well-posed problems; for linear equations, a compatibility condition yields an exact projection characterization. For a scalar conservation law with transonic boundary data, every minimizing sequence converges to a continuous function that does not solve the equation, even when a stationary entropy shock exists. We give conditions under which approaching this limit forces the parameter norms to diverge. A Hamilton--Jacobi example shows that a function can have zero residual without being the viscosity solution. Favorable results for a viscous model and uniformly convex Monge--Ampère critical points identify conditions that restore the corresponding implications. For piecewise-smooth trial families, the regularity needed to define the residual functional does not by itself justify the sampled gradient used in training. Moving residual jumps can contribute terms absent from that gradient; an explicit example has identically vanishing sampled gradients and a nonzero continuous derivative. Together, these results clarify what stationarity and sampled gradients imply about the residual, and under which conditions minimizing sequences lead to the intended solution.
ER -