\chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation
Abstract
Finding all modes of a multimodal black-box function is a fundamental challenge in optimization, Bayesian inference, and scientific computing. Existing approaches -- basin-hopping, CMA-ES, multistart gradient descent -- operate sequentially and cannot exploit the massive parallelism of modern GPU hardware. We introduce \chisao{} (\textbf{C}onvergence-\textbf{H}alt-\textbf{I}nvert-\textbf{S}tick-\textbf{A}nd-\textbf{O}scillate), a GPU-native population optimizer that runs an entire sample batch simultaneously and exploits a deliberate convergence-anticonvergence oscillation cycle to escape local traps while freezing confirmed modes. The structural move is asymmetric: samples that reach true peaks are frozen (``stuck'') and preserved, while the rest keep exploring via momentum-based anti-convergence and stochastically smoothed gradients. Adaptive reseeding via two complementary strategies (Repulse Monkey and Golden Rooster) maintains population diversity throughout. On all 42 functions of the Simon Fraser University optimization benchmark suite across dimensions d ∈ {2, 4, 8, 16, 32, 64}, \chisao{} achieves \textbf{100\%} mode recovery where all CPU baselines collapse at d ≥ 8 on the hardest multimodal functions, at up to \textbf{34×} speedup over basin-hopping on functions where all methods succeed (Michalewicz d=64) and up to \textbf{39×} on unimodal functions (Rotated Hyper-Ellipsoid d=64, pure GPU dividend). All benchmarks evaluate the objective by value alone -- gradients come from finite differences -- so the reported speedups are a derivative-free worst case. Under substantial likelihood noise (σnoise up to 1.0), mode detection remains 100\% reliable. The algorithm is available as a standalone open-source Python package on PyPI.
Versions
Related
- Convergence rates for the RMSprop optimizer with full control of the hyperparameters
- Uniform oscillation properties of almost periodic functions
- The MCMC convergence graph: a diagnostic for multimodal posteriors
- On some bounds for cardinal invariants of the space of quasicontinuous functions with the topology of pointwise convergence
- Rapid convergence of Langevin AIS for multimodal distributions on ℝd
BibTeX
@misc{wolfson2026chisao,
title = {{\chisao: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation}},
author = {Ira Wolfson},
howpublished = {arXiv},
year = {2026},
month = jun,
doi = {10.48550/arxiv.2606.26164},
eprint = {2606.26164},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2606.26164},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - \chisao{}: A GPU-Native Parallel Optimizer for Multimodal Black-Box Functions via Convergence-Anticonvergence Oscillation
AU - Ira Wolfson
PY - 2026
DA - 2026/06/24
PB - arXiv
DO - 10.48550/arxiv.2606.26164
UR - https://arxiv.org/abs/2606.26164
AB - Finding all modes of a multimodal black-box function is a fundamental challenge in optimization, Bayesian inference, and scientific computing. Existing approaches -- basin-hopping, CMA-ES, multistart gradient descent -- operate sequentially and cannot exploit the massive parallelism of modern GPU hardware. We introduce \chisao{} (\textbf{C}onvergence-\textbf{H}alt-\textbf{I}nvert-\textbf{S}tick-\textbf{A}nd-\textbf{O}scillate), a GPU-native population optimizer that runs an entire sample batch simultaneously and exploits a deliberate convergence-anticonvergence oscillation cycle to escape local traps while freezing confirmed modes. The structural move is asymmetric: samples that reach true peaks are frozen (``stuck'') and preserved, while the rest keep exploring via momentum-based anti-convergence and stochastically smoothed gradients. Adaptive reseeding via two complementary strategies (Repulse Monkey and Golden Rooster) maintains population diversity throughout. On all 42 functions of the Simon Fraser University optimization benchmark suite across dimensions d ∈ {2, 4, 8, 16, 32, 64}, \chisao{} achieves \textbf{100\%} mode recovery where all CPU baselines collapse at d ≥ 8 on the hardest multimodal functions, at up to \textbf{34×} speedup over basin-hopping on functions where all methods succeed (Michalewicz d=64) and up to \textbf{39×} on unimodal functions (Rotated Hyper-Ellipsoid d=64, pure GPU dividend). All benchmarks evaluate the objective by value alone -- gradients come from finite differences -- so the reported speedups are a derivative-free worst case. Under substantial likelihood noise (σ_noise up to 1.0), mode detection remains 100\% reliable. The algorithm is available as a standalone open-source Python package on PyPI.
ER -