MpSub: A Momentum p-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
Abstract
Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by estimating update directions from loss evaluations, but existing methods require tuning a sensitive learning rate for each model and task. We propose the momentum p-dimensional subspace trust-region method (MpSub). At each iteration, MpSub searches within a p-dimensional subspace: one direction preserves historical momentum from the most recent accepted step, while the remaining directions explore via fresh random sampling. The subspace gradient is estimated by central differences, a trial step is computed from a linear trust-region model, and the trust-region radius adapts according to the agreement between predicted and observed loss reduction, eliminating the learning rate. For LLM fine-tuning, evaluations within an iteration share a minibatch, and directions are regenerated in place from seeds, using forward passes alone. For smooth deterministic objectives under unorthogonalized Gaussian directions, we bound the finite-difference error, quantify gradient energy captured by the subspace, and prove that limk→∞ |∇ f(xk)|2 = 0 almost surely under a safeguarded radius update. Under a matched budget of 8,400 training-objective forward passes, we fine-tune OPT-125M and OPT-350M on CommitmentBank. With the same preset parameters at both model sizes, MpSub attains mean test accuracies of 0.673 and 0.690 over three seeds, matching tuned MeZO (0.685) without any learning-rate search.
Related
- Random-subspace Ellipsoidal Trust-region Methods for Derivative-free Optimization
- A note on the complexity of random subspace model-based methods for derivative-free optimization
- A particle swarm optimization with a reduced-subspace trust-region safeguard for contact-rich controller tuning
- A two-dimensional subspace of ℓ∞3 has a unique operator space structure if and only if it is isometric to ℓ∞2
- An Iterative Active Subspace Approach for Model Order Reduction of Parametric Systems with High-Dimensional Parameter Spaces
BibTeX
@misc{wang2026mpsub,
title = {{MpSub: A Momentum $p$-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models}},
author = {Yuyang Wang and Haoyu Yao and Pengcheng Xie},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.07666},
eprint = {2609.07666},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.07666},
}
RIS
TY - UNPB
TI - MpSub: A Momentum p-Dimensional Subspace Trust-Region Method for Derivative-Free Fine-Tuning of Large Language Models
AU - Yuyang Wang
AU - Haoyu Yao
AU - Pengcheng Xie
PY - 2026
DA - 2026/09/07
PB - arXiv
DO - 10.48550/arxiv.2609.07666
UR - https://arxiv.org/abs/2609.07666
AB - Full-parameter fine-tuning of large language models has substantial memory costs because backpropagation stores activations and gradients. Zeroth-order optimization avoids this by estimating update directions from loss evaluations, but existing methods require tuning a sensitive learning rate for each model and task. We propose the momentum p-dimensional subspace trust-region method (MpSub). At each iteration, MpSub searches within a p-dimensional subspace: one direction preserves historical momentum from the most recent accepted step, while the remaining directions explore via fresh random sampling. The subspace gradient is estimated by central differences, a trial step is computed from a linear trust-region model, and the trust-region radius adapts according to the agreement between predicted and observed loss reduction, eliminating the learning rate. For LLM fine-tuning, evaluations within an iteration share a minibatch, and directions are regenerated in place from seeds, using forward passes alone. For smooth deterministic objectives under unorthogonalized Gaussian directions, we bound the finite-difference error, quantify gradient energy captured by the subspace, and prove that lim_k→∞ |∇ f(x_k)|_2 = 0 almost surely under a safeguarded radius update. Under a matched budget of 8,400 training-objective forward passes, we fine-tune OPT-125M and OPT-350M on CommitmentBank. With the same preset parameters at both model sizes, MpSub attains mean test accuracies of 0.673 and 0.690 over three seeds, matching tuned MeZO (0.685) without any learning-rate search.
ER -