Skip to content
DraconiticResearch
Search Sources API
181,716 studies31 sourcesUpdated 55 min ago
PreprintCC BY 4.0arXiv

Automated Numerical Stability Analysis of Deep Learning Operators

  • Xinye Chen
Open at sourcePDFDOICite

Abstract

Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to numerical instability. In this paper, we introduce a unified software tool for stochastic numerical validation of deep-learning operators. The tool follows CESTAC on supported exposed operations and uses an operator-level data-perturbation approximation for GEMM-like kernels. Our developed software not only enables numerical validation with a single computation pass but also detects the sources of numerical instability and provides numerical stability monitoring during deep learning training and inference. We verified its effectiveness on the detection of polluted operators with injected numerical instabilities across various tasks. We believe that our developed method and tools provide valuable insights into developing numerically stable computing kernels, which are particularly critical for numerically stable and efficient deep learning training and inference.

Related

  1. Numerical Neural Operator: Connection Between Numerical Analysis and the Operator Learning and Building Discretized Neural Operators from Finite Element Methods
    Zecheng Zhang
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingarXiv
  2. Spectral and numerical radius preservers on pairs of unitarily similar operators
    Zouheir Amara
    arXivPreprintarXiv licenceFull textAnalysis & PDEsarXiv
  3. Numerical range and invariant subspaces of weighted composition operators on the Hardy space of Dirichlet series
    Sudeshna Lahiri, Sarita Ojha, Subhadip Halder +1
    arXivPreprintarXiv licenceFull textAnalysis & PDEsarXiv
  4. Some Generalized Numerical Ranges of Real-linear Operators on Hilbert Spaces
    Arup Majumdar, Vladimir Muller
    arXivPreprintCC BY-NC-SA 4.0Full textAnalysis & PDEsarXiv
  5. Sibuya probability distribution and numerical evaluation of fractional-order operators
    Nikolai Leonenko, Igor Podlubny
    Fractional Calculus & Applied AnalysisPeer-reviewedCC BY 4.0Full text1 citedGeneral mathematicsPMCEurope PMC

Cite

BibTeXRIS
BibTeX
@misc{chen2026automated,
  title = {{Automated Numerical Stability Analysis of Deep Learning Operators}},
  author = {Xinye Chen},
  howpublished = {arXiv},
  year = {2026},
  month = nov,
  doi = {10.48550/arxiv.2607.25494},
  eprint = {2607.25494},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2607.25494},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - Automated Numerical Stability Analysis of Deep Learning Operators
AU  - Xinye Chen
PY  - 2026
DA  - 2026/11/03
PB  - arXiv
DO  - 10.48550/arxiv.2607.25494
UR  - https://arxiv.org/abs/2607.25494
AB  - Finite-precision arithmetic unavoidably introduces numerical approximation errors. Numerical computations may use insufficient precision or an improper formulation, which leads to numerical instability. In this paper, we introduce a unified software tool for stochastic numerical validation of deep-learning operators. The tool follows CESTAC on supported exposed operations and uses an operator-level data-perturbation approximation for GEMM-like kernels. Our developed software not only enables numerical validation with a single computation pass but also detects the sources of numerical instability and provides numerical stability monitoring during deep learning training and inference. We verified its effectiveness on the detection of polluted operators with injected numerical instabilities across various tasks. We believe that our developed method and tools provide valuable insights into developing numerically stable computing kernels, which are particularly critical for numerically stable and efficient deep learning training and inference.
ER  -