{
  "abstract": "Population loss can remain nearly constant while a neural network learns a substantially more predictive representation. We establish this separation for two-layer ReLU and leaky-ReLU networks trained on Gaussian inputs by simultaneous fixed-step population gradient descent on all parameters. For structured additive teachers whose links are positive mixtures of Gaussian-damped cubics in $H^1(γ)$, we give explicit conditions under which small IID Gaussian initialization yields a high-probability guarantee: at a checkpoint during a high-loss plateau, minimum alignment between the rank-$r$ teacher subspace and the leading $r$-dimensional eigenspace of the predictor's average gradient outer product (AGOP) increases by at least $1/2$, and the minimum refit MSE under unchanged coefficient budgets decreases by more than $0.399$, both relative to initialization. The same trajectory subsequently attains a trained loss below every value in the plateau window. A complementary result treats unequal-weight cubic teachers and small additive Sobolev perturbations using projected-feature refits. For SwiGLU networks with an exactly fitted intercept, we prove leading-AGOP alignment during a loss plateau at fixed width and dimension as Gaussian initialization vanishes, for square-integrable teachers with nonzero Hermite content of degree one, two, or three. A rank-one cubic specialization also gives simultaneous unrestricted-refit gains at a prescribed width. An approximation lower bound further shows that certain interaction targets retain nonzero error when ridge neurons are restricted to shared orthogonal axes within the teacher subspace. Population-moment experiments with ReLU students across 21 teachers and 50 initializations per teacher complement the analysis.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2609.39408",
  "author_total": 1,
  "authors": [
    {
      "name": "Akash Kumar"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/23j061nnn8j6r.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2609.39408",
  "fulltext_url": null,
  "id": "23j061nnn8j6r",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2609.39408",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "math.OC"
    ],
    [
      "arxiv",
      "stat.ML"
    ],
    [
      "openalex",
      "1702|Stochastic Gradient Optimization Techniques"
    ]
  ],
  "openalex": "W7215033858",
  "pdf_url": "https://arxiv.org/pdf/2609.39408",
  "pmcid": null,
  "pmid": null,
  "published": "2026-09-30",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/23j061nnn8j6r.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:25:49Z",
      "native_id": "2609.39408",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2609.39408"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:math.OC",
      "subfield": "optimisation-control"
    },
    {
      "field": "mathematics",
      "origin": "arxiv:stat.ML",
      "subfield": "statistics"
    }
  ],
  "title": "Awakening of the Buddha: Subspace Learning During Population-Loss Plateaus",
  "title_text": "Awakening of the Buddha: Subspace Learning During Population-Loss Plateaus",
  "type": "preprint",
  "updated_at": "2026-10-30T12:25:58Z",
  "url": "https://research.draconitic.com/paper/23j061nnn8j6r",
  "venue": "arXiv",
  "versions": []
}