Skip to content
DraconiticResearch
Search Sources API
203,234 studies31 sourcesUpdated 50 min ago
PreprintCC BY 4.0arXiv

Fast Differentiable SVD on GPU via Polar Decomposition

  • Uliana Parkina
  • Askar Tsyganov
  • Sergei Kudriashov
  • Sergey Samsonov
  • Maxim Rakhuba
Open at sourcePDFDOICite

Abstract

We present a fully GPU-oriented SVD pipeline based on polar decomposition, motivated by iterative methods that rely solely on matrix multiplications, such as the Newton-Schulz iteration. We show that this approach enables up to a 2× speedup compared to standard implementations. Furthermore, we derive a numerically stable backward pass for the polar decomposition and leverage it to obtain a fully differentiable SVD. Our methods are released as open-source implementations in both PyTorch and JAX: https://github.com/fallnlove/cans_svd.

Related

  1. Reversed and Refined Berezin Number Inequalities via Buzano’s Inequality and Polar Decomposition
    Reem K. Alhefthi, Silvestru Sever Dragomir
    MathematicsPeer-reviewedCC BY 4.0General mathematicsCrossref
  2. Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
    Sayantan Choudhury, Xiaoran Cheng, Martin Takáč +2
    arXivPreprintarXiv licenceFull textOptimisation & controlarXiv
  3. Polar Switch: Fast Muon Orthogonalization with Polynomial Circuits
    Zhehang Du, Weijie Su
    arXivPreprintCC BY 4.0Full textNumerical analysis & scientific computingOptimisation & controlarXiv
  4. A Fast Algorithm for Quaternion Tensor Singular Value Decomposition under the QT-Product with Applications to Color Video Processing
    Daochang Zhang, Yue Zhao, Jingqian Li
    arXivPreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
  5. Extended dynamic mode decomposition with Fourier dictionaries: Error bounds and fast implementation
    Felix Bartel, Sandra Ritter, Manuel Schaller +1
    arXivPreprintCC BY-NC-ND 4.0Full textDynamical systemsarXiv

Cite

BibTeXRIS
BibTeX
@misc{parkina2026fast,
  title = {{Fast Differentiable SVD on GPU via Polar Decomposition}},
  author = {Uliana Parkina and Askar Tsyganov and Sergei Kudriashov and Sergey Samsonov and Maxim Rakhuba},
  howpublished = {arXiv},
  year = {2026},
  month = sep,
  doi = {10.48550/arxiv.2609.32505},
  eprint = {2609.32505},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.32505},
  note = {CC BY 4.0},
}
RIS
TY  - UNPB
TI  - Fast Differentiable SVD on GPU via Polar Decomposition
AU  - Uliana Parkina
AU  - Askar Tsyganov
AU  - Sergei Kudriashov
AU  - Sergey Samsonov
AU  - Maxim Rakhuba
PY  - 2026
DA  - 2026/09/26
PB  - arXiv
DO  - 10.48550/arxiv.2609.32505
UR  - https://arxiv.org/abs/2609.32505
AB  - We present a fully GPU-oriented SVD pipeline based on polar decomposition, motivated by iterative methods that rely solely on matrix multiplications, such as the Newton-Schulz iteration. We show that this approach enables up to a 2× speedup compared to standard implementations. Furthermore, we derive a numerically stable backward pass for the polar decomposition and leverage it to obtain a fully differentiable SVD. Our methods are released as open-source implementations in both PyTorch and JAX: https://github.com/fallnlove/cans_svd.
ER  -