{
  "abstract": "Engine self-play allows chess openings to be compared under controlled conditions, without the selection effects of human game databases. When openings differ only slightly, however, rankings built from a few hundred games can be unstable. In Study 1, strength-limited Stockfish 15.1 played 17,400 games: ten forced White first moves and ten forced Black replies to 1.e4, each at three strength settings. The leading opening changed between settings, twice by a single game. There was no statistically reliable evidence, though, that differences among openings depended on strength (White p = .311, Black p = .843). The design had limited sensitivity to isolated strength-dependent effects: a 12-point effect was detected by the omnibus test in only about 39% of simulated studies. A Connect Four positive control detected an observed 22.5-point center-column advantage, demonstrating sensitivity to large opening differences. With 290 games per opening, the observed leader was the true best of ten openings in only about 44% of simulated studies assuming a between-opening standard deviation of three percentage points. Follow-up studies with complete game records showed that the settings were ordered in direct matches but did not form a consistent Elo scale, that win and draw rates differed across hardware and implementation configurations, and that an apparent 1.d3 weakness was not confirmed in new games. With Stockfish 19 and a fixed node budget, survey-selected replies showed no statistically reliable advantage over free engine choice. Opening leaderboards from a few hundred engine games can overstate certainty.",
  "abstract_shown": true,
  "affiliations": [],
  "article_type": null,
  "arxiv": "2610.37988",
  "author_total": 4,
  "authors": [
    {
      "name": "Conrad Flick"
    },
    {
      "name": "Jaysen Jensen"
    },
    {
      "name": "Jingyu Huang"
    },
    {
      "name": "Mario Camara Paslar"
    }
  ],
  "bibtex": "https://research.draconitic.com/paper/3g3sc6eg5312s.bib",
  "cited_by": 0,
  "doi": "10.48550/arxiv.2610.37988",
  "fulltext_url": null,
  "id": "3g3sc6eg5312s",
  "issns": [],
  "landing_url": "https://arxiv.org/abs/2610.37988",
  "language": null,
  "licence": "CC-BY-4.0",
  "licence_label": "CC BY 4.0",
  "licence_source": "article",
  "licence_url": "http://creativecommons.org/licenses/by/4.0/",
  "metadata_licence": "CC0-1.0",
  "native_subjects": [
    [
      "arxiv",
      "stat.AP"
    ]
  ],
  "openalex": null,
  "pdf_url": "https://arxiv.org/pdf/2610.37988",
  "pmcid": null,
  "pmid": null,
  "published": "2026-10-27",
  "publisher": null,
  "ris": "https://research.draconitic.com/paper/3g3sc6eg5312s.ris",
  "sources": [
    {
      "first_seen": "2026-10-30T12:36:41Z",
      "native_id": "2610.37988",
      "source": "arxiv",
      "url": "https://arxiv.org/abs/2610.37988"
    }
  ],
  "subjects": [
    {
      "field": "mathematics",
      "origin": "arxiv:stat.AP",
      "subfield": "statistics"
    }
  ],
  "title": "Opening Rankings from Engine Self-Play Are Less Certain Than They Look: Sampling Error, Draws, and Strength Dependence in Strength-Limited Stockfish Games",
  "title_text": "Opening Rankings from Engine Self-Play Are Less Certain Than They Look: Sampling Error, Draws, and Strength Dependence in Strength-Limited Stockfish Games",
  "type": "preprint",
  "updated_at": "2026-11-01T12:21:00Z",
  "url": "https://research.draconitic.com/paper/3g3sc6eg5312s",
  "venue": "arXiv",
  "versions": []
}