Measuring Learned Equivariance: An Information Theoretic Approach
Abstract
Learning symmetries through data augmentation offers an alternative to architectural equivariance, but does not guarantee that a network respects those symmetries. Measuring learned equivariance is especially difficult for intermediate representations, whose group actions are not specified. In a finite discrete setting, we show that equivariance requires representations of original and transformed inputs to determine one another exactly, yielding a test based on normalized mutual information (NMI). For high-dimensional activations, we use centered kernel alignment (CKA) as a scalable test, with an exact guarantee for orthogonal feature transformations. Across architectures and tasks, augmentation usually increases transformation consistency, showing that learned equivariance can be probed directly from representations without recovering the hidden transformation rules that relate them.
Related
- Measuring Learned Monotone Temporal Aggregation at Matched Admissibility
- A Sample-Based Approach for Hierarchical Information-Theoretic Compression of Probabilistic Occupancy Grids
- Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences
- Learning Rules, Memorizing Facts: An Information-Theoretic View of Hallucination
- Efficient Robust Learning at the Information-Theoretic Limit
BibTeX
@misc{schlesinger2026measuring,
title = {{Measuring Learned Equivariance: An Information Theoretic Approach}},
author = {Claire Schlesinger and Wolfgang Gatterbauer and Robin Walters},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.17269},
eprint = {2610.17269},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.17269},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Measuring Learned Equivariance: An Information Theoretic Approach
AU - Claire Schlesinger
AU - Wolfgang Gatterbauer
AU - Robin Walters
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2610.17269
UR - https://arxiv.org/abs/2610.17269
AB - Learning symmetries through data augmentation offers an alternative to architectural equivariance, but does not guarantee that a network respects those symmetries. Measuring learned equivariance is especially difficult for intermediate representations, whose group actions are not specified. In a finite discrete setting, we show that equivariance requires representations of original and transformed inputs to determine one another exactly, yielding a test based on normalized mutual information (NMI). For high-dimensional activations, we use centered kernel alignment (CKA) as a scalable test, with an exact guarantee for orthogonal feature transformations. Across architectures and tasks, augmentation usually increases transformation consistency, showing that learned equivariance can be probed directly from representations without recovering the hidden transformation rules that relate them.
ER -