Skip to content
DraconiticResearch
Search Sources API
226,742 studies31 sourcesUpdated 40 min ago
PreprintCC BY 4.0arXiv

Fast and Stable Gradient Approximation for Quadratic Forms of Hermitian Matrix Functions

Abstract

Objectives involving quadratic forms u*f(A(θ))u for Hermitian A arise widely in scientific computing and machine learning. Lanczos efficiently approximates these quantities, but differentiating through its recurrence can introduce numerical instability and costly reorthogonalization. Building on existing low-rank Krylov approximations of Fréchet derivatives, we develop a matrix-free gradient implementation that reuses the forward Lanczos quantities without differentiating through the recurrence. We analyze its discrepancy from the exact gradient of the finite Lanczos approximation, deriving an exact residual-coupled error identity and a bound involving the best degree-(m−1) uniform polynomial approximation error of f', multiplied by a basis-sensitivity factor. The discrepancy vanishes when the Krylov subspace is invariant. Experiments demonstrate accurate gradients and runtime improvements over differentiated-Lanczos and modified-Arnoldi baselines. It remains numerically stable without reorthogonalization in all tested settings, including cases where adjoint-based Lanczos differentiation becomes unstable.

Related

  1. Stable Rational Approximation of PDE Transfer Functions with H∞ Error Bounds
    Aleksandr Talitckii, Matthew M. Peet
    arXivPreprintarXiv licenceFull textAnalysis & PDEsarXiv
  2. Hermitian K-theory for stable ∞-categories III: Grothendieck--Witt groups of rings
    Baptiste Calmès, Emanuele Dotto, Yonatan Harpaz +6
    Annals of MathematicsPeer-reviewedLicence not statedFull text57 citedMathematical physicsOpenAlex
  3. A Galois-type Springer theorem for hermitian and generalized quadratic forms
    Amir Hossein Nokhodkar
    arXivPreprintarXiv licenceFull textAlgebra & number theoryarXiv
  4. Stable moduli spaces of Hermitian forms
    Fabian Hebestreit, Wolfgang Steimle, Yonatan Harpaz
    Forum of Mathematics PiPeer-reviewedCC BY-NC-ND37 citedMathematical physicsOpenAlex
  5. Stability and gradient estimates for fully nonlinear elliptic equations on hermitian manifolds
    Bin Guo, Jian Song
    arXivPreprintCC BY 4.0Full textAnalysis & PDEsGeometry & topologyarXiv
BibTeX
@misc{singh2026fast,
  title = {{Fast and Stable Gradient Approximation for Quadratic Forms of Hermitian Matrix Functions}},
  author = {Navjot Singh and Kipton Barros and Xiaoye Sherry Li},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2605.12801},
  eprint = {2605.12801},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.12801},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - Fast and Stable Gradient Approximation for Quadratic Forms of Hermitian Matrix Functions
AU  - Navjot Singh
AU  - Kipton Barros
AU  - Xiaoye Sherry Li
PY  - 2026
DA  - 2026/11/07
PB  - arXiv
DO  - 10.48550/arxiv.2605.12801
UR  - https://arxiv.org/abs/2605.12801
AB  - Objectives involving quadratic forms u^*f(A(θ))u for Hermitian A arise widely in scientific computing and machine learning. Lanczos efficiently approximates these quantities, but differentiating through its recurrence can introduce numerical instability and costly reorthogonalization. Building on existing low-rank Krylov approximations of Fréchet derivatives, we develop a matrix-free gradient implementation that reuses the forward Lanczos quantities without differentiating through the recurrence. We analyze its discrepancy from the exact gradient of the finite Lanczos approximation, deriving an exact residual-coupled error identity and a bound involving the best degree-(m−1) uniform polynomial approximation error of f', multiplied by a basis-sensitivity factor. The discrepancy vanishes when the Krylov subspace is invariant. Experiments demonstrate accurate gradients and runtime improvements over differentiated-Lanczos and modified-Arnoldi baselines. It remains numerically stable without reorthogonalization in all tested settings, including cases where adjoint-based Lanczos differentiation becomes unstable.
ER  -