How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
Abstract
Recently, Morris et al. introduced TinyLoRA, a strategy for LLM fine-tuning which introduces arbitrarily few parameters, and demonstrated accuracy improvements using less than $10$ parameters rather than the ≳ 10,000 typically demanded by LoRA-based methods. However, that study continued to use gradient-based local search methods of the kind deployed for training multi-billion parameter LLMs in the first place. This article is predicated on the observation that, in the low dimensional setting, global optimization methods become tractable, unlocking Large language Model Bayesian Optimization (LlMBO). LlMBO is sample-efficient, gradient-free, memory-light, parallelizes across devices without synchronization, and, unlike policy-gradient methods, can optimize greedy-decoding accuracy directly. We describe numerical experiments comparing LlMBO against GRPO on fine-tuning tasks for a variety of models on a math benchmark suite which show that LlMBO can be a suitable alternative in this regime, matching or exceeding GRPO's fine tuning gains, while using a fraction of the compute and memory.
Related
- ProLand: A New Infeasible Manifold Optimization Method for LoRA Fine-Tuning
- Statistical Benefits of Fine-Tuning from Pretrained Initialization in Diagonal Linear Networks
- Two-Timescale Fine-tuning Provably Learns New Features for Two-Layer ReLU Networks
- Retrieval-Guided Fine-Tuning as Noisy Estimation: Risk bounds and Architectural Analysis
- No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning Collapse
BibTeX
@misc{kulkarni2026finetuning,
title = {{How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)}},
author = {Aditya Kulkarni and Davide Sartor and Carlos Soto and Nathan Wycoff},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.18504},
eprint = {2610.18504},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.18504},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - How Low Can You Go: Fine-Tuning LLMs with Bayesian Optimization (LlMBO)
AU - Aditya Kulkarni
AU - Davide Sartor
AU - Carlos Soto
AU - Nathan Wycoff
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.18504
UR - https://arxiv.org/abs/2610.18504
AB - Recently, Morris et al. introduced TinyLoRA, a strategy for LLM fine-tuning which introduces arbitrarily few parameters, and demonstrated accuracy improvements using less than $10$ parameters rather than the ≳ 10,000 typically demanded by LoRA-based methods. However, that study continued to use gradient-based local search methods of the kind deployed for training multi-billion parameter LLMs in the first place. This article is predicated on the observation that, in the low dimensional setting, global optimization methods become tractable, unlocking Large language Model Bayesian Optimization (LlMBO). LlMBO is sample-efficient, gradient-free, memory-light, parallelizes across devices without synchronization, and, unlike policy-gradient methods, can optimize greedy-decoding accuracy directly. We describe numerical experiments comparing LlMBO against GRPO on fine-tuning tasks for a variety of models on a math benchmark suite which show that LlMBO can be a suitable alternative in this regime, matching or exceeding GRPO's fine tuning gains, while using a fraction of the compute and memory.
ER -