Can a search hold its own bar? In-loop deflation and its exploits in autonomous strategy discovery
Abstract
Corrections for backtest overfitting such as the deflated Sharpe ratio are applied after a search ends: the searcher maximises raw performance and a separate procedure deflates the reported winner. Autonomous searches driven by large language models perform many implicit backtests before reporting, so this paper asks what changes when the correction moves inside the loop and the acceptance threshold is indexed to the running count and dispersion of the search's own trials. The threshold then becomes an object the search can optimise against: we show analytically that a naive in-loop bar admits a variance-dilution effect, in which mediocre near-duplicate candidates lower the bar because the ledger's dispersion falls faster than its trial-count factor grows. We propose two defences, a pre-registered variance floor and a monotone high-water-mark bar, and evaluate the construction under a protocol frozen before any run, with every in-loop decision replaying exactly from its recorded trace. Across 53 audited hundred-candidate searches over two frontier models, two universes, and two feature spaces, the gated search reports "no certifiable discovery" on 36 of 37 runs, un-gated controls report a strategy on all 16, and every arm's out-of-sample confidence interval includes zero. Supporting studies find that state-based stopping reaches the same outcome after 8 to 9 trials at about a twelfth of the trial budget while an adaptive budget adds nothing, that a controlled stress test reproduces the dilution effect and both defences, and that a hundred nominal trials collapse to 6 to 8 effectively independent return streams. We scope these results to the analysed system.
Related
- GRAFT-ATHENA: Self-Improving Agentic Teams for Autonomous Discovery and Evolutionary Numerical Algorithms
- Autonomous Structural Discovery Of Bioprocess Dynamics And Kinetics Via LLM-Driven Agents
- SiteEvo: An LLM Agent for Autonomous Discovery of Site-Adaptive Massive MIMO Detectors
- Large-Scale Autonomous Discovery of Kissing Number Constructions
- Kähler-Ricci Flow is Symplectic Deflation plus an Exact Symplectic Deformation
BibTeX
@misc{gencay2026search,
title = {{Can a search hold its own bar? In-loop deflation and its exploits in autonomous strategy discovery}},
author = {Eray Gençay},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.14539},
eprint = {2611.14539},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.14539},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Can a search hold its own bar? In-loop deflation and its exploits in autonomous strategy discovery
AU - Eray Gençay
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.14539
UR - https://arxiv.org/abs/2611.14539
AB - Corrections for backtest overfitting such as the deflated Sharpe ratio are applied after a search ends: the searcher maximises raw performance and a separate procedure deflates the reported winner. Autonomous searches driven by large language models perform many implicit backtests before reporting, so this paper asks what changes when the correction moves inside the loop and the acceptance threshold is indexed to the running count and dispersion of the search's own trials. The threshold then becomes an object the search can optimise against: we show analytically that a naive in-loop bar admits a variance-dilution effect, in which mediocre near-duplicate candidates lower the bar because the ledger's dispersion falls faster than its trial-count factor grows. We propose two defences, a pre-registered variance floor and a monotone high-water-mark bar, and evaluate the construction under a protocol frozen before any run, with every in-loop decision replaying exactly from its recorded trace. Across 53 audited hundred-candidate searches over two frontier models, two universes, and two feature spaces, the gated search reports "no certifiable discovery" on 36 of 37 runs, un-gated controls report a strategy on all 16, and every arm's out-of-sample confidence interval includes zero. Supporting studies find that state-based stopping reaches the same outcome after 8 to 9 trials at about a twelfth of the trial budget while an adaptive budget adds nothing, that a controlled stress test reproduces the dilution effect and both defences, and that a hundred nominal trials collapse to 6 to 8 effectively independent return streams. We scope these results to the analysed system.
ER -