Linear Regression Beyond Covariance: Latent Slope Spectra from Higher-Order Cumulants
Abstract
Least squares summarizes the joint distribution of a response and regressor through second-order information. When an observed regressor is composed of latent components carrying different relationships with the response, this summary can average over distinct linear effects. We study the additional linear information contained in higher-order mixed cumulants and formalize it through the latent slope spectrum. Ordinary least squares is a variance-weighted barycenter of this spectrum, whereas higher-order cumulants can separate its components when the spectrum is sufficiently parsimonious. In the scalar one-confounder model, fourth-order cumulants identify the unordered pair {β,β+γ} through a 2×2 linear system and a quadratic equation. We develop estimation and root-n inference under regular separation and show that detection and estimation diverge near degeneracy: slope multiplicity remains detectable on an n−1/4 local scale even when the individual slopes are not uniformly regularly estimable. In multiple regression, a quadratic tensor recurrence yields a test of one versus multiple effective coefficient vectors that does not require the vectors themselves to be identified; multiplicity can therefore be consistently detectable even when the recurrence operator is noninjective. The same quadratic diagnostic can retain signal in directions where a first-order higher-cumulant restriction cancels exactly. The results distinguish representation, identification, detection, and estimation of latent linear heterogeneity, while making explicit the limits of structural and causal interpretation from observational data alone.
Related
- Covariance spectra of random linear systems near instability
- Linear statistics in the Coulomb gas: cumulants beyond rotational invariance
- Linear Exponential Quadratic Gaussian Covariance Steering
- Score-based confidence intervals for variance-covariance parameters in linear mixed models
- Tests for Categorical Data Beyond Pearson: A Distance Covariance and Energy Distance Approach
BibTeX
@misc{nath2026linear,
title = {{Linear Regression Beyond Covariance: Latent Slope Spectra from Higher-Order Cumulants}},
author = {Anirban Nath and Ming Yuan},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.17772},
eprint = {2610.17772},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.17772},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Linear Regression Beyond Covariance: Latent Slope Spectra from Higher-Order Cumulants
AU - Anirban Nath
AU - Ming Yuan
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.17772
UR - https://arxiv.org/abs/2610.17772
AB - Least squares summarizes the joint distribution of a response and regressor through second-order information. When an observed regressor is composed of latent components carrying different relationships with the response, this summary can average over distinct linear effects. We study the additional linear information contained in higher-order mixed cumulants and formalize it through the latent slope spectrum. Ordinary least squares is a variance-weighted barycenter of this spectrum, whereas higher-order cumulants can separate its components when the spectrum is sufficiently parsimonious. In the scalar one-confounder model, fourth-order cumulants identify the unordered pair {β,β+γ} through a 2×2 linear system and a quadratic equation. We develop estimation and root-n inference under regular separation and show that detection and estimation diverge near degeneracy: slope multiplicity remains detectable on an n^−1/4 local scale even when the individual slopes are not uniformly regularly estimable. In multiple regression, a quadratic tensor recurrence yields a test of one versus multiple effective coefficient vectors that does not require the vectors themselves to be identified; multiplicity can therefore be consistently detectable even when the recurrence operator is noninjective. The same quadratic diagnostic can retain signal in directions where a first-order higher-cumulant restriction cancels exactly. The results distinguish representation, identification, detection, and estimation of latent linear heterogeneity, while making explicit the limits of structural and causal interpretation from observational data alone.
ER -