Normalization Couples What the Objective Separates: Representation Selection in Group Composition Networks
- 1University of Nevada, Las Vegas
Abstract
We study when the representation selected by a normalized network can be predicted by scoring candidate representations independently. The model is a two-layer quadratic network on a finite group’s multiplication table, with real Fourier sectors as the candidates. At small logit scale the leading objective is additive across sectors, but the sphere-projection multiplier depends on the sum of all sector terms. A sector-local rule scores each candidate from its own Fourier coefficients and every sector’s per-layer magnitudes, but not from a rival’s signs, phases or matrix directions. Let the joint input sphere have metric weight ω and the output sphere weight one; ordinary normalization has ω = 1. For groups with at least three nontrivial real sectors, sector-local rules recover the leading selector almost everywhere if and only if ω = 2, the degree-matched weight. At every other fixed weight the optimal sector-local error under uniform initialization is positive: reflecting the output block of a third sector that wins at neither start fixes both rivals’ scores yet switches the winner.
Under ordinary normalization, at fixed group and width, exact cross entropy almost surely inherits the leading labels as the scale vanishes. At width one, with three distinct nontrivial real linear characters, the optimal sector-local error of continuous exact cross entropy is positive at every sufficiently small scale, even at ω = 2, where it tends to zero with the scale. Retuning the weight in a compact set does not remove it. On two-bit XOR under ordinary normalization, a simple sector-local rule errs on 1.7% of sampled starts. There the jointly and separately normalized leading selectors have identical label frequencies but disagree on an estimated 18% of paired starts. All results concern representation selection at fixed parameter norms, not loss, accuracy or generalization.
Related
- Representation, normalization and architecture interact across operating- and hardware-domain shifts in source-only fault diagnosis
- Newly Important Is Not Newly Built: What Reasoning Post-Training Adds Inside a Language Model, Why the Evidence Separates Representation From Wiring From Deployment, and the Cross-Model Tests That Would Show an Attention Head Is New
- Dependency-aware evolutionary search using population representation learning for large-scale many-objective optimisation
- Imagine No Resources: Attention Is Selection and Normalization for Choice
- Enhancing the accuracy of qPCR normalization in Bactrocera zonata: A systematic study for the selection of condition-optimized reference genes for gene expression studies
BibTeX
@misc{sehrawat2026normalization,
title = {{Normalization Couples What the Objective Separates: Representation Selection in Group Composition Networks}},
author = {Vanshdeep Sehrawat},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23650374},
url = {https://zenodo.org/records/23650374},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Normalization Couples What the Objective Separates: Representation Selection in Group Composition Networks
AU - Vanshdeep Sehrawat
PY - 2026
DA - 2026/10/25
PB - Zenodo
DO - 10.5281/zenodo.23650374
UR - https://zenodo.org/records/23650374
AB - We study when the representation selected by a normalized network can be predicted by scoring candidate representations independently. The model is a two-layer quadratic network on a finite group’s multiplication table, with real Fourier sectors as the candidates. At small logit scale the leading objective is additive across sectors, but the sphere-projection multiplier depends on the sum of all sector terms. A sector-local rule scores each candidate from its own Fourier coefficients and every sector’s per-layer magnitudes, but not from a rival’s signs, phases or matrix directions. Let the joint input sphere have metric weight ω and the output sphere weight one; ordinary normalization has ω = 1. For groups with at least three nontrivial real sectors, sector-local rules recover the leading selector almost everywhere if and only if ω = 2, the degree-matched weight. At every other fixed weight the optimal sector-local error under uniform initialization is positive: reflecting the output block of a third sector that wins at neither start fixes both rivals’ scores yet switches the winner. Under ordinary normalization, at fixed group and width, exact cross entropy almost surely inherits the leading labels as the scale vanishes. At width one, with three distinct nontrivial real linear characters, the optimal sector-local error of continuous exact cross entropy is positive at every sufficiently small scale, even at ω = 2, where it tends to zero with the scale. Retuning the weight in a compact set does not remove it. On two-bit XOR under ordinary normalization, a simple sector-local rule errs on 1.7% of sampled starts. There the jointly and separately normalized leading selectors have identical label frequencies but disagree on an estimated 18% of paired starts. All results concern representation selection at fixed parameter norms, not loss, accuracy or generalization.
ER -