Riemannian Optimization for Hadamard Products of Low-Rank Matrices
Abstract
The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.
Related
- Bounds on the Minimum Eigenvalue Modulus for Hadamard Products of M- and H-Matrices and Their Inverses
- Riemannian Structure and Optimization for a Class of Low-Parametric Orthogonal Matrices
- Riemannian optimization on low-rank tensor train manifolds for the Gross-Pitaevskii equation
- Complex Hadamard matrices and strongly regular graphs
- Classification of Symmetric Hadamard Matrices Up to Order 32
BibTeX
@misc{jawanpuria2026riemannian,
title = {{Riemannian Optimization for Hadamard Products of Low-Rank Matrices}},
author = {Pratik Jawanpuria and Ankish Chandresh and Bamdev Mishra},
howpublished = {arXiv},
year = {2026},
month = aug,
doi = {10.48550/arxiv.2606.01216},
eprint = {2606.01216},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.01216},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Riemannian Optimization for Hadamard Products of Low-Rank Matrices
AU - Pratik Jawanpuria
AU - Ankish Chandresh
AU - Bamdev Mishra
PY - 2026
DA - 2026/08/31
PB - arXiv
DO - 10.48550/arxiv.2606.01216
UR - https://arxiv.org/abs/2606.01216
AB - The elementwise Hadamard product of two low-rank matrices provides a parameter-efficient model for data with multiplicative structure, but its modeling is challenging due to the presence of additional symmetries under coupled row/column scalings between the two factors. In order to leverage the geometry of the space, we formulate the learning of such matrices as optimization on a Riemannian quotient manifold. We propose a novel block-diagonal Riemannian metric derived from the pullback of the Frobenius inner product. The metric is shown to be invariant under these symmetries. We develop a Riemannian gradient descent algorithm that uses a tuning-free Gauss--Newton step size and scales linearly in the number of observed entries per iteration. The versatile framework of Riemannian quotient optimization enables both first-order and second-order Riemannian methods, the latter through a closed-form connection and the Riemannian Hessian. Experiments on real and synthetic datasets illustrate the efficacy of our proposed Riemannian approach.
ER -