Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models

Abstract

We establish a polynomial sample complexity separation between symmetry-aware and symmetry-agnostic feature learning. We study growing-rank multi-index models with high-dimensional Gaussian covariates in ℝ^d and r=Θ(d^δ) teacher directions forming a cyclic symmetry orbit, where 0<δ<1/2. We compare three ways of exploiting this structure: architectural weight sharing, data augmentation over the full symmetry group, and learning without access to the symmetry. In particular, we analyze a symmetry-tied convolutional network, an untied network, and the same untied network trained with full-group data augmentation, using spherical online SGD with correlation loss. For a class of polynomial links with information exponent p≥3, we prove matching sample complexity bounds up to logarithmic factors: the tied and augmented learners achieve weak directional recovery in Θ̃(d^p−1) samples, whereas the symmetry-agnostic learner requires Θ̃(rd^p−1). For the pure quadratic Hermite link, the same separation holds for weak recovery of the teacher subspace, with sample complexities Θ̃(d) and Θ̃(rd), respectively. Thus, full-group data augmentation matches the sample efficiency of architectural weight sharing, and both provide a polynomial advantage over training without symmetry. For p≥3, the proof reveals a two-stage mechanism: fluctuations at initialization select one direction in the teacher orbit, after which localized growth amplifies its overlap to the weak recovery scale while competing overlaps remain near their initialization scale.

Elliptic Fekete points: separation, Riesz energies, and polynomial conditioning
Ujué Etayo
arXiv30 Oct 2026PreprintCC BY-NC-ND 4.0Full textAnalysis & PDEsarXiv
Superintegrability of discrete-time rational Ruijsenaars-Schneider model and deformed polynomial symmetry algebras
Pavel Drozdov
arXiv1 Oct 2026PreprintarXiv licenceFull textMathematical physics Statistical & nonlinear physicsarXiv
Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions
Dier Tang, Guangyue Han
arXiv15 Sep 2026PreprintarXiv licenceFull textStatisticsarXiv
Adaptive Randomized Neural Networks for PDEs: Structure-Aware Feature Redistribution and Residual-Driven Block Enrichment
You Yang, Fei Wang
arXiv15 Oct 2026PreprintarXiv licenceFull textNumerical analysis & scientific computingarXiv
Bet on Features: Anytime-Valid and Feature-Aware Auditing of Conditional Quantile Forecasters
Ivane Antonov, Sohom Mukherjee, Richard Pibernik +1
arXiv28 Oct 2026PreprintCC BY 4.0Full textStatisticsarXiv

Cite

BibTeX RIS

BibTeX

@misc{waber2026symmetryaware,
  title = {{Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models}},
  author = {Jivan Waber and Vanessa Piccolo and Yatin Dandi and Florent Krzakala},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2610.08420},
  eprint = {2610.08420},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.08420},
}

RIS

TY  - UNPB
TI  - Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models
AU  - Jivan Waber
AU  - Vanessa Piccolo
AU  - Yatin Dandi
AU  - Florent Krzakala
PY  - 2026
DA  - 2026/10/06
PB  - arXiv
DO  - 10.48550/arxiv.2610.08420
UR  - https://arxiv.org/abs/2610.08420
AB  - We establish a polynomial sample complexity separation between symmetry-aware and symmetry-agnostic feature learning. We study growing-rank multi-index models with high-dimensional Gaussian covariates in ℝ^d and r=Θ(d^δ) teacher directions forming a cyclic symmetry orbit, where 0<δ<1/2. We compare three ways of exploiting this structure: architectural weight sharing, data augmentation over the full symmetry group, and learning without access to the symmetry. In particular, we analyze a symmetry-tied convolutional network, an untied network, and the same untied network trained with full-group data augmentation, using spherical online SGD with correlation loss. For a class of polynomial links with information exponent p≥3, we prove matching sample complexity bounds up to logarithmic factors: the tied and augmented learners achieve weak directional recovery in Θ̃(d^p−1) samples, whereas the symmetry-agnostic learner requires Θ̃(rd^p−1). For the pure quadratic Hermite link, the same separation holds for weak recovery of the teacher subspace, with sample complexities Θ̃(d) and Θ̃(rd), respectively. Thus, full-group data augmentation matches the sample efficiency of architectural weight sharing, and both provide a polynomial advantage over training without symmetry. For p≥3, the proof reveals a two-stage mechanism: fluctuations at initialization select one direction in the teacher orbit, after which localized growth amplifies its overlap to the weak recovery scale while competing overlaps remain near their initialization scale.
ER  -

Abstract

Related

Cite