The Equivariance Criterion in a Linear Model for Random-X Cases
Abstract
Equivariance is increasingly used in machine learning and statistics, often without systematic justification. In a companion article, the equivariance criterion was applied to the normal linear model with a fixed design matrix (fixed-X), yielding the minimum risk equivariant (MRE) estimators of the coefficient vector and of the condensed diagonal covariance matrix under a multivariate invariant location--scale group. We extend these results to the random-X case, with covariates sampled from a population. The extension hinges on a distinction vacuous for fixed-X but fundamental for random-X: whether risk and unbiasedness are evaluated conditionally on the realized design or after averaging over the design distribution. Under conditional evaluation, the fixed-X group applies given X: least squares remains the best equivariant estimator of the coefficient vector, and the MRE estimators of the population variances keep their fixed-X forms with population sizes at the realized design. Under absolute evaluation with an i.i.d.\ design, the picture changes qualitatively: the natural scale group acting jointly on (Y,X) fixes the coefficient vector, the induced parameter-space action is intransitive, equivariant risks are constant only along orbits indexed by the signal-to-noise ratio ρ=|β|2/σ2, and no uniformly minimum risk equivariant estimator exists. In the scalar case the optimal equivariant weight is the oracle shrinkage factor w*(ρ)=ρ/(ρ+E[T−1]), with least squares recovered as the infinite-signal limit ρ→∞---explaining and refining the known failure of the Gauss--Markov theorem with random regressors. For a centered design under location--scale transformations, least squares remains optimal within the natural invariant-contrast class, and the MRE estimator S2/(n−p+2) of the error variance is valid under both modes.
Related
- Two-sided estimates for two-point local time tails of branching random walks: a Green function criterion
- A linear independence criterion for p-adic Hurwitz zeta values at odd integers
- The higher-dimensional Shepp problem: an exact criterion for random ball coverings of tori
- Smooth Information Criterion for Variable Selection in Generalised Linear Models
- On the Convergence of the Mesh-Free Discretization of a Non-Linear Bond-Based Peridynamic Model with a Bond-Breakage Criterion
BibTeX
@misc{zhang2026equivariance,
title = {{The Equivariance Criterion in a Linear Model for Random-$X$ Cases}},
author = {Zhengyang Zhang and Haojin Zhou},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.16897},
eprint = {2609.16897},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.16897},
}
RIS
TY - UNPB
TI - The Equivariance Criterion in a Linear Model for Random-X Cases
AU - Zhengyang Zhang
AU - Haojin Zhou
PY - 2026
DA - 2026/09/15
PB - arXiv
DO - 10.48550/arxiv.2609.16897
UR - https://arxiv.org/abs/2609.16897
AB - Equivariance is increasingly used in machine learning and statistics, often without systematic justification. In a companion article, the equivariance criterion was applied to the normal linear model with a fixed design matrix (fixed-X), yielding the minimum risk equivariant (MRE) estimators of the coefficient vector and of the condensed diagonal covariance matrix under a multivariate invariant location--scale group. We extend these results to the random-X case, with covariates sampled from a population. The extension hinges on a distinction vacuous for fixed-X but fundamental for random-X: whether risk and unbiasedness are evaluated conditionally on the realized design or after averaging over the design distribution. Under conditional evaluation, the fixed-X group applies given X: least squares remains the best equivariant estimator of the coefficient vector, and the MRE estimators of the population variances keep their fixed-X forms with population sizes at the realized design. Under absolute evaluation with an i.i.d.\ design, the picture changes qualitatively: the natural scale group acting jointly on (Y,X) fixes the coefficient vector, the induced parameter-space action is intransitive, equivariant risks are constant only along orbits indexed by the signal-to-noise ratio ρ=|β|^2/σ^2, and no uniformly minimum risk equivariant estimator exists. In the scalar case the optimal equivariant weight is the oracle shrinkage factor w^*(ρ)=ρ/(ρ+E[T^−1]), with least squares recovered as the infinite-signal limit ρ→∞---explaining and refining the known failure of the Gauss--Markov theorem with random regressors. For a centered design under location--scale transformations, least squares remains optimal within the natural invariant-contrast class, and the MRE estimator S^2/(n−p+2) of the error variance is valid under both modes.
ER -