Why can genetic algorithms work in high-dimensional search spaces?
Abstract
We show that the effective dynamics of the elitist (1+M) genetic algorithm is, in the limit of small mutations, normalized gradient descent on the loss in the presence of anisotropic Gaussian white noise. In expectation, therefore, a simple mutation-selection genetic algorithm follows the gradient of the loss, without explicit calculation of gradients and without averaging over loss evaluations. The genetic algorithm is slower than gradient descent because of the noise that acts in directions transverse to the gradient. However, this slowdown is controlled not by the number of parameters of the search space but by the effective rank of the Hessian of the loss function. For the concentrated Hessian spectra observed in neural-network loss functions the effective rank can be far smaller than the number of parameters, which may explain why genetic algorithms can scale to large search spaces.
Related
- Optimal Quantum Algorithms for Ordered Search
- Circuit-depth optimization of quantum partial-search algorithms
- When is global evolutionary search useful for variational quantum algorithms? A landscape-first study
- A literature-guided descriptor-based framework for filtering composition search spaces
- Three-channel dissipative warm Higgs inflation with global inference via genetic algorithms
BibTeX
@misc{whitelam2026genetic,
title = {{Why can genetic algorithms work in high-dimensional search spaces?}},
author = {Stephen Whitelam},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2606.30619},
eprint = {2606.30619},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.30619},
}
RIS
TY - UNPB
TI - Why can genetic algorithms work in high-dimensional search spaces?
AU - Stephen Whitelam
PY - 2026
DA - 2026/09/28
PB - arXiv
DO - 10.48550/arxiv.2606.30619
UR - https://arxiv.org/abs/2606.30619
AB - We show that the effective dynamics of the elitist (1+M) genetic algorithm is, in the limit of small mutations, normalized gradient descent on the loss in the presence of anisotropic Gaussian white noise. In expectation, therefore, a simple mutation-selection genetic algorithm follows the gradient of the loss, without explicit calculation of gradients and without averaging over loss evaluations. The genetic algorithm is slower than gradient descent because of the noise that acts in directions transverse to the gradient. However, this slowdown is controlled not by the number of parameters of the search space but by the effective rank of the Hessian of the loss function. For the concentrated Hessian spectra observed in neural-network loss functions the effective rank can be far smaller than the number of parameters, which may explain why genetic algorithms can scale to large search spaces.
ER -