Escaping the Capacity Ceiling: Routing on the Stiefel Manifold for Bilinear SPD Layers
Abstract
Deep networks on the symmetric positive-definite (SPD) manifold promise expressive representations by encoding data geometry as an inductive bias, but stacking BiMap layers with the standard ReEig nonlinearity often adds no capacity: on real, preconditioned EEG data, ReEig rarely activates, so the stack behaves as a single layer at any depth. In the worst case, when domains share no discriminative directions, we prove a single filter has a capacity ceiling, so it cannot fully align every domain at once. To overcome that, we propose SCAP (Stiefel Cross-Attention Pool), a layer implementing a family of Stiefel filters by combining a pool of K experts into a sample-specific bilinear map via cross-attention. We show that it matches a per-domain filter bank to first order with fewer experts than domains when domain-optimal filters span few directions near a shared tangent-space basepoint; in the worst case, its alignment empirically stays nearly flat as domains grow, escaping the fixed-filter ceiling. Naively trained, however, this routing can collapse to a fixed filter; we diagnose why and adapt three mechanisms to mitigate it. SCAP significantly improves balanced accuracy over fixed-filter SPDNet on all five cross-domain EEG motor-imagery datasets, and matches or exceeds three domain-adaptive baselines on four out of five.
Related
- Directions That Don't Drift: Stiefel Manifold Routing for Transformer Attention
- The BergmanPoisson Subharmonic Bridge and PickLoewner Bilinear Coercivity (Part III): Coboundary Pinning at z = 1, Green Capacity Bounds, and O-Diagonal Phase Dominance
- One-bubble solution with maximum points escaping to infinity for the energy-critical heat equation in ℝ5
- Escaping and generics over ideals
- On the Hausdorff dimension of the set of non-escaping points in the Julia set
BibTeX
@misc{maia2026escaping,
title = {{Escaping the Capacity Ceiling: Routing on the Stiefel Manifold for Bilinear SPD Layers}},
author = {Isabella C. Maia and Salem Said and Pedro L. C. Rodrigues and Marco Congedo},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.31043},
eprint = {2605.31043},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.31043},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Escaping the Capacity Ceiling: Routing on the Stiefel Manifold for Bilinear SPD Layers
AU - Isabella C. Maia
AU - Salem Said
AU - Pedro L. C. Rodrigues
AU - Marco Congedo
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2605.31043
UR - https://arxiv.org/abs/2605.31043
AB - Deep networks on the symmetric positive-definite (SPD) manifold promise expressive representations by encoding data geometry as an inductive bias, but stacking BiMap layers with the standard ReEig nonlinearity often adds no capacity: on real, preconditioned EEG data, ReEig rarely activates, so the stack behaves as a single layer at any depth. In the worst case, when domains share no discriminative directions, we prove a single filter has a capacity ceiling, so it cannot fully align every domain at once. To overcome that, we propose SCAP (Stiefel Cross-Attention Pool), a layer implementing a family of Stiefel filters by combining a pool of K experts into a sample-specific bilinear map via cross-attention. We show that it matches a per-domain filter bank to first order with fewer experts than domains when domain-optimal filters span few directions near a shared tangent-space basepoint; in the worst case, its alignment empirically stays nearly flat as domains grow, escaping the fixed-filter ceiling. Naively trained, however, this routing can collapse to a fixed filter; we diagnose why and adapt three mechanisms to mitigate it. SCAP significantly improves balanced accuracy over fixed-filter SPDNet on all five cross-domain EEG motor-imagery datasets, and matches or exceeds three domain-adaptive baselines on four out of five.
ER -