Adaptive Partitioning Schemes for Optimistic Optimization
Abstract
Applications such as engineering design often require us to optimize a black-box function, i.e., a system whose inner processing is not analytically known and whose gradients are not available. Practitioners often have a fixed budget for the number of function evaluations and the performance of an optimization algorithm is measured by its simple regret. In this paper, we study the class of "Optimistic Optimization" algorithms for black-box optimization that use a partitioning scheme for the domain. We develop algorithms that learn a good partitioning scheme and use flexible surrogate models such as neural networks in the optimization procedure. For multi-index functions on an m-dimensional subspace within d dimensions, our algorithm attains Õ(n−β/ d) regret, where β= 1 + (d−m)/(2m−1), as opposed to Õ(n−1/d) for SequOOL, a state-of-the-art optimistic optimization algorithm. We use our approach to improve the quality of Activation-aware Weight Quantization (AWQ) of the OPT-1.3B model, achieving ∼10% improvement in performance relative to the best possible unquantized model.
Related
- A simple adaptive accelerated optimistic dual averaging method for unconstrained convex optimization in p-norm geometry
- Adaptive Local Proper Orthogonal Decomposition via Graph Partitioning and Provable Error-Bounded Lossy Compression for Parametric Series in Finite-Dimensional Function Spaces
- Are implicit schemes needed in adaptive time stepping for PDEs?
- Iterative coupling schemes for mixed-dimensional poromechanical models with frictional contact: convergence analysis and adaptive stabilization
- Linear adaptive variable time step schemes for the anisotropic phase field dendritic crystal growth model
BibTeX
@misc{sunkara2026adaptive,
title = {{Adaptive Partitioning Schemes for Optimistic Optimization}},
author = {Raja Sunkara and Ardhendu Tripathy},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.04039},
eprint = {2610.04039},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.04039},
}
RIS
TY - UNPB
TI - Adaptive Partitioning Schemes for Optimistic Optimization
AU - Raja Sunkara
AU - Ardhendu Tripathy
PY - 2026
DA - 2026/10/02
PB - arXiv
DO - 10.48550/arxiv.2610.04039
UR - https://arxiv.org/abs/2610.04039
AB - Applications such as engineering design often require us to optimize a black-box function, i.e., a system whose inner processing is not analytically known and whose gradients are not available. Practitioners often have a fixed budget for the number of function evaluations and the performance of an optimization algorithm is measured by its simple regret. In this paper, we study the class of "Optimistic Optimization" algorithms for black-box optimization that use a partitioning scheme for the domain. We develop algorithms that learn a good partitioning scheme and use flexible surrogate models such as neural networks in the optimization procedure. For multi-index functions on an m-dimensional subspace within d dimensions, our algorithm attains Õ(n^−β/ d) regret, where β= 1 + (d−m)/(2m−1), as opposed to Õ(n^−1/d) for SequOOL, a state-of-the-art optimistic optimization algorithm. We use our approach to improve the quality of Activation-aware Weight Quantization (AWQ) of the OPT-1.3B model, achieving ∼10% improvement in performance relative to the best possible unquantized model.
ER -