On batch normalization for SPDnet
Abstract
This paper deals with batch normalization for SPDnet, a deep learning architecture specifically designed to handle covariance matrices. Current SPDnet batch normalization relies on geometric means, which require expensive iterative computations and lacks formal backpropagation derivations. These limitations are addressed through two contributions: (i) formal backpropagation derivation for SPDnet batch normalization, and (ii) computationally efficient closed-form alternatives to the geometric mean: log-Euclidean, arithmetic and harmonic means, and the geometric mean of arithmetic and harmonic means. Simulation results on the batch normalization layer with different means demonstrate significant reductions in memory consumption and comparable training time by avoiding automatic differentiation, demonstrating the effectiveness of our approach. Numerical experiments on three real datasets show that simpler means can outperform the geometric mean with up to 5% accuracy improvements while reducing training time by a factor of 5.
Related
- Explicit diagonalization of the linearized Calderón problem on the disk: multiplicity normalization and exponential spectral attenuation
- Self-normalization for Spectral Density Integrals
- Automorphic Transcendence Degree, Growth, and Skew Normalization
- Normalization by Evaluation with Substitutions as Operations
- Dold--Kan Correspondences via Morita Normalization
BibTeX
@article{gallet2026batch,
title = {{On batch normalization for SPDnet}},
author = {Matthieu Gallet and Ammar Mian and Florent Bouchard and Guillaume Ginolhac},
journal = {Signal Processing},
year = {2026},
month = jul,
doi = {10.1016/j.sigpro.2026.110793},
url = {https://hal.science/hal-05742151v1},
note = {CC BY 4.0},
}
RIS
TY - JOUR
TI - On batch normalization for SPDnet
AU - Matthieu Gallet
AU - Ammar Mian
AU - Florent Bouchard
AU - Guillaume Ginolhac
PY - 2026
DA - 2026/07/16
JO - Signal Processing
DO - 10.1016/j.sigpro.2026.110793
UR - https://hal.science/hal-05742151v1
AB - This paper deals with batch normalization for SPDnet, a deep learning architecture specifically designed to handle covariance matrices. Current SPDnet batch normalization relies on geometric means, which require expensive iterative computations and lacks formal backpropagation derivations. These limitations are addressed through two contributions: (i) formal backpropagation derivation for SPDnet batch normalization, and (ii) computationally efficient closed-form alternatives to the geometric mean: log-Euclidean, arithmetic and harmonic means, and the geometric mean of arithmetic and harmonic means. Simulation results on the batch normalization layer with different means demonstrate significant reductions in memory consumption and comparable training time by avoiding automatic differentiation, demonstrating the effectiveness of our approach. Numerical experiments on three real datasets show that simpler means can outperform the geometric mean with up to 5% accuracy improvements while reducing training time by a factor of 5.
SN - 0165-1684
SN - 1872-7557
ER -