When and Why Do Linear Bias Probes Fail? A Geometric and Statistical Theory of Bias Detectability in Large Language Model Representations
Abstract
Linear probing is the standard instrument for detecting social biases in the hidden representations of large language models. Yet reported probe accuracies come almost exclusively from \emph{counterfactual} evaluations in which every input carries an explicit demographic marker. Once only a fraction α of inputs carries demographic information, performance degrades sharply, and a weak probe may reflect either an unbiased model or an underpowered detector. We develop a theory that resolves this ambiguity. Modeling representations as two class-conditional clusters with Mahalanobis separation s on a manifold of curvature kap, we prove: (i) a finite-sample generalization bound governed by the manifold's extrinsic radius with a matching smash√(dB/n) minimax lower bound; (ii) an exact purity law for the maximum linear-probe AUC, strictly increasing in α; (iii) a curvature ceiling: ambient chordal separation on a space form cannot exceed 2/√(kap); and (iv) a detectability threshold below which no audit can distinguish probe output from chance. Every theorem is validated on synthetic manifolds with known ground truth and on six open-weight models × four bias dimensions, where the purity law predicts entire AUC--α curves from a single cross-fitted ŝ measured at α=1, with no parameters fitted to those curves. The framework turns bias auditing into a power analysis: given a target purity and effect size, it prescribes the sample budget n(α) for a conclusive audit.
Related
- From detectability of abstract linear systems to exponential output-to-state stability and back
- A geometric construction of supercuspidal representations
- Descriptor representations of geometric port-Hamiltonian systems on Hilbert spaces
- Information-Geometric Grammar Principle: A Statistical-Geometric Framework for Adaptive Systems
- Reducibility of linear representations, free ideals, and Kippenhahn's conjecture
BibTeX
@misc{hai2026when,
title = {{When and Why Do Linear Bias Probes Fail? A Geometric and Statistical Theory of Bias Detectability in Large Language Model Representations}},
author = {Mo Hai and Haifeng Li},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.22337},
eprint = {2609.22337},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22337},
}
RIS
TY - UNPB
TI - When and Why Do Linear Bias Probes Fail? A Geometric and Statistical Theory of Bias Detectability in Large Language Model Representations
AU - Mo Hai
AU - Haifeng Li
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.22337
UR - https://arxiv.org/abs/2609.22337
AB - Linear probing is the standard instrument for detecting social biases in the hidden representations of large language models. Yet reported probe accuracies come almost exclusively from \emph{counterfactual} evaluations in which every input carries an explicit demographic marker. Once only a fraction α of inputs carries demographic information, performance degrades sharply, and a weak probe may reflect either an unbiased model or an underpowered detector. We develop a theory that resolves this ambiguity. Modeling representations as two class-conditional clusters with Mahalanobis separation s on a manifold of curvature kap, we prove: (i) a finite-sample generalization bound governed by the manifold's extrinsic radius with a matching smash√(dB/n) minimax lower bound; (ii) an exact purity law for the maximum linear-probe AUC, strictly increasing in α; (iii) a curvature ceiling: ambient chordal separation on a space form cannot exceed 2/√(kap); and (iv) a detectability threshold below which no audit can distinguish probe output from chance. Every theorem is validated on synthetic manifolds with known ground truth and on six open-weight models × four bias dimensions, where the purity law predicts entire AUC--α curves from a single cross-fitted ŝ measured at α=1, with no parameters fitted to those curves. The framework turns bias auditing into a power analysis: given a target purity and effect size, it prescribes the sample budget n(α) for a conclusive audit.
ER -