Chance-Constrained Personalized LLM Routing with Online Preference Learning
Abstract
We study personalized LLM routing, where a router selects among heterogeneous models with different costs and capabilities. The router must minimize cumulative inference cost while ensuring reliable realized response quality, even though user preferences are unknown and must be learned from feedback during deployment. We formulate this problem as a constrained contextual bandit and develop an algorithm that learns preferences while balancing cost and quality. Under a logistic preference model and a strict-feasibility condition, we establish sublinear expected cost regret relative to an optimal stationary randomized contextual oracle and a high-probability guarantee on realized cumulative satisfaction. The framework extends beyond LLM routing to broader online model-selection problems with reliability requirements.
Related
- Distribution-Agnostic Robust Trajectory Optimization via Chance-Constrained Reinforcement Learning
- PREFER: Personalized Review Summarization with Online Preference Learning
- Chance-Constrained Inspection Viewpoint Sequencing for On-Orbit Servicing via Safe Reinforcement Learning
- An Intersection Cutting Plane Algorithm for Chance-Constrained Programs with Finite Support
- Chance-Constrained Nonlinear Covariance Control via Robust Linearization Remainder Bounds
BibTeX
@misc{anjarlekar2026chanceconstrained,
title = {{Chance-Constrained Personalized LLM Routing with Online Preference Learning}},
author = {Ameya Anjarlekar and Rasoul Etesami and R Srikant},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.22695},
eprint = {2610.22695},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.22695},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Chance-Constrained Personalized LLM Routing with Online Preference Learning
AU - Ameya Anjarlekar
AU - Rasoul Etesami
AU - R Srikant
PY - 2026
DA - 2026/10/15
PB - arXiv
DO - 10.48550/arxiv.2610.22695
UR - https://arxiv.org/abs/2610.22695
AB - We study personalized LLM routing, where a router selects among heterogeneous models with different costs and capabilities. The router must minimize cumulative inference cost while ensuring reliable realized response quality, even though user preferences are unknown and must be learned from feedback during deployment. We formulate this problem as a constrained contextual bandit and develop an algorithm that learns preferences while balancing cost and quality. Under a logistic preference model and a strict-feasibility condition, we establish sublinear expected cost regret relative to an optimal stationary randomized contextual oracle and a high-probability guarantee on realized cumulative satisfaction. The framework extends beyond LLM routing to broader online model-selection problems with reliability requirements.
ER -