The Fourth Quadrant: A Stylized View of Benign Misfitting

Abstract

Training error is what we can observe on a training set; test error is the quantity we care about. We study squared-error linear regression in a deterministic (d+1)-dimensional single-spike model. Each training vector has spike coordinate √γ, γ>1, and equal-norm, mutually orthogonal nuisance components. Training labels are $1$. Fresh test points are centered Gaussian with covariance diag(γ,1,…,1) and noise-free label x[1]/√γ. We compute the exact train--test tradeoff for linear predictors in the training span. When the spike direction is unknown, allowing all linear predictors does not improve this tradeoff, on average or in the worst case over directions consistent with the data. There is a range of training-set sizes where every span predictor that generalizes well must fit the training data worse than the zero predictor. We call this benign misfitting, or the fourth quadrant. At its sharpest, test error tends to zero while training error diverges, with both zero-predictor errors normalized to one. The best span predictor begins to generalize when n≫ d/γ², while interpolation needs n≫ d/γ. In the intervening window d/γ²≪ n≪ d/γ, useful span prediction lies beyond interpolation: training predictions overshoot the labels. One-pass stochastic gradient descent with a large constant learning rate attains small test error throughout this window, matching the best span predictor up to a logarithmic factor, while leaving large empirical training error. Finally, the nuisance component responsible for the training misfit also controls adversarial sensitivity.

The Weighted Walks in Quadrant with Finite Groups: an Algebro-Geometric Approach
Vladimir Dragovic, Milena Radnovic
arXiv4 Sep 2026PreprintarXiv licenceFull textAlgebra & number theory Combinatorics & discrete mathematicsarXiv
Quadrant classification and development path of gymnastics-related research topics in Chinese high-quality journals: Based on SCI, PKU Core and CSSCI source journals
Jingli Yu, Wen Sun, Luping Qu +3
PLOS One2 Nov 2026Peer-reviewedCC BYFull textGeneral biology StatisticsPLOSEurope PMCPMC
A remark on the fourth order Q curvature on manifolds with dimension at least 5
Fengbo Hang
arXiv30 Aug 2026PreprintCC BY 4.0Full textGeometry & topologyarXiv
L^p-regularity for fourth-order Schrödinger operators with unbounded or degenerate coefficients
Federica Gregorio, Chiara Spina, Cristian Tacelli
arXiv16 Oct 2026PreprintarXiv licenceFull textAnalysis & PDEsarXiv
Spectral fourth moments of Hecke--Maaß cusp forms
Edgar Assing, Peter Humphries
arXiv15 Jul 2026PreprintarXiv licenceFull textAlgebra & number theoryarXiv

Cite

BibTeX RIS

BibTeX

@misc{ranade2026fourth,
  title = {{The Fourth Quadrant: A Stylized View of Benign Misfitting}},
  author = {Gireeja Ranade and Anant Sahai},
  howpublished = {arXiv},
  year = {2026},
  month = oct,
  doi = {10.48550/arxiv.2608.01032},
  eprint = {2608.01032},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.01032},
}

RIS

TY  - UNPB
TI  - The Fourth Quadrant: A Stylized View of Benign Misfitting
AU  - Gireeja Ranade
AU  - Anant Sahai
PY  - 2026
DA  - 2026/10/08
PB  - arXiv
DO  - 10.48550/arxiv.2608.01032
UR  - https://arxiv.org/abs/2608.01032
AB  - Training error is what we can observe on a training set; test error is the quantity we care about. We study squared-error linear regression in a deterministic (d+1)-dimensional single-spike model. Each training vector has spike coordinate √γ, γ>1, and equal-norm, mutually orthogonal nuisance components. Training labels are $1$. Fresh test points are centered Gaussian with covariance diag(γ,1,…,1) and noise-free label x[1]/√γ. We compute the exact train--test tradeoff for linear predictors in the training span. When the spike direction is unknown, allowing all linear predictors does not improve this tradeoff, on average or in the worst case over directions consistent with the data. There is a range of training-set sizes where every span predictor that generalizes well must fit the training data worse than the zero predictor. We call this benign misfitting, or the fourth quadrant. At its sharpest, test error tends to zero while training error diverges, with both zero-predictor errors normalized to one. The best span predictor begins to generalize when n≫ d/γ^2, while interpolation needs n≫ d/γ. In the intervening window d/γ^2≪ n≪ d/γ, useful span prediction lies beyond interpolation: training predictions overshoot the labels. One-pass stochastic gradient descent with a large constant learning rate attains small test error throughout this window, matching the best span predictor up to a logarithmic factor, while leaving large empirical training error. Finally, the nuisance component responsible for the training misfit also controls adversarial sensitivity.
ER  -

Abstract

Related

Cite