{
  "abstract": "Prior Fitted Networks (PFNs) such as TabPFN now rival established statistical procedures across prediction and estimation tasks. A natural explanation is that PFNs have the property of statistical adaptivity, that is, they perform nearly as well as a method tailored to the true data-generating model for a heterogeneous set of models, while not being told which model the data comes from. We study how such adaptivity is learned in a controlled location-estimation problem. Each task is an unlabeled sample whose family is hidden: Gaussian data call for averaging, with error of order $n^{-1}$, whereas uniform data are best estimated from their extremes, at the faster rate $n^{-2}$. We also provide the example of a symmetric Gaussian mixture, for which a rate of $σ^2_n/n$ can be attained. On scalar inputs, softmax attention computes the derivative of the empirical cumulant-generating function. A single primitive therefore both supplies features that distinguish the families and forms estimators interpolating between the sample mean and the mid-range. We combine attention experts through either a softmax mixture of experts or a gated linear unit (GLU), and analyze stagewise gradient flow. With $\\widetildeΩ(n^{1+ε})$ pretraining tasks, the learned estimator is asymptotically efficient on Gaussian tasks, within a factor $n^ε$ of the minimax rate on uniform tasks, and order-optimal on mixtures in a shrinking-variance regime. These guarantees extend to new locations and longer contexts. A risk decomposition separates expert error, routing error and normalization error, which clarifies the architectural contrast. Softmax gating enforces normalization and exact translation equivariance, whereas the GLU must learn it: its dynamics separate into fast bias removal followed by slow expert selection. End-to-end experiments recover the predicted specialization.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2610.07804",
  "author_total": 5,
  "authors": [
    {
      "name": "Martin Eppert"
    },
    {
      "name": "Krishna Balasubramanian"
    },
    {
      "name": "Subhro Ghosh"
    },
    {
      "name": "Jason Klusowski"
    },
    {
      "name": "Yan Shuo Tan"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/ce45hsw5872md.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2610.07804",
  "fulltext_url": null,
  "id": "ce45hsw5872md",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2610.07804",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "cs.LG"
    ],
    [
      "arxiv",
      "stat.ML"
    ],
    [
      "openalex",
      "1702|Neural Networks and Applications"
    ]
  ],
  "openalex": "W7221042629",
  "pdf_url": "https://arxiv.org/pdf/2610.07804",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-06",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/ce45hsw5872md.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:27:32Z",
      "native_id": "2610.07804",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2610.07804"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:stat.ML",
      "subfield": "statistics"
    }
  ],
  "title": "Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis",
  "title_text": "Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis",
  "type": "preprint",
  "updated_at": "2026-10-30T12:27:32Z",
  "url": "https://research.draconitic.com/paper/ce45hsw5872md",
  "venue": "arXiv",
  "versions": []
}