Learning in Continuous Games from Pairwise Preference Feedback
Abstract
We study learning in continuous games when players receive only pairwise preference feedback, revealing which of two actions is preferred but neither payoff values nor preference magnitudes. We first show that standard external regret and coarse correlated equilibria (CCE) are not identifiable from this ordinal information: the same sequence of play can incur zero and linear regret in two ordinally equivalent games, while the distributions that remain CCE across all cardinal representations consistent with the same preferences are exactly those supported on pure Nash equilibria. Motivated by this gap, we develop a first-order ordinal theory based on normalized unilateral preference directions, introducing an ordinal directional regret benchmark and corresponding equilibrium notions. We show that block-normalized pseudogradient dynamics achieve sublinear ordinal regret and, under additional structure, Nash-convergence guarantees. We then use a single-comparison estimator to implement these dynamics from finite pairwise comparisons. With one comparison per player and round, the resulting algorithm achieves sublinear finite-resolution ordinal regret against arbitrary opponent behavior and, in ordinal potential games, almost-sure last-iterate convergence to the Nash set. Our results provide regret, dynamics, and equilibrium guarantees directly from preference feedback without reconstructing cardinal utilities.
Related
- Continuous-Time Reinforcement Learning for N-Player Stochastic Differential Games with Exploratory Policies
- Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments
- Continuous Envy-Free Allocation via Reciprocal Preference Matrices: An Alternative to Sperner-Based Discretization
- Horizon-Independent Contraction for Continuous-Time Discounted Regularized Mean-Field Games
- Multidimensional Infinite-Horizon Markovian Backward Stochastic Differential Equations with Continuous Coefficients and Nonzero-Sum Differential Games
BibTeX
@misc{barakat2026learning,
title = {{Learning in Continuous Games from Pairwise Preference Feedback}},
author = {Anas Barakat},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.05428},
eprint = {2610.05428},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.05428},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Learning in Continuous Games from Pairwise Preference Feedback
AU - Anas Barakat
PY - 2026
DA - 2026/10/04
PB - arXiv
DO - 10.48550/arxiv.2610.05428
UR - https://arxiv.org/abs/2610.05428
AB - We study learning in continuous games when players receive only pairwise preference feedback, revealing which of two actions is preferred but neither payoff values nor preference magnitudes. We first show that standard external regret and coarse correlated equilibria (CCE) are not identifiable from this ordinal information: the same sequence of play can incur zero and linear regret in two ordinally equivalent games, while the distributions that remain CCE across all cardinal representations consistent with the same preferences are exactly those supported on pure Nash equilibria. Motivated by this gap, we develop a first-order ordinal theory based on normalized unilateral preference directions, introducing an ordinal directional regret benchmark and corresponding equilibrium notions. We show that block-normalized pseudogradient dynamics achieve sublinear ordinal regret and, under additional structure, Nash-convergence guarantees. We then use a single-comparison estimator to implement these dynamics from finite pairwise comparisons. With one comparison per player and round, the resulting algorithm achieves sublinear finite-resolution ordinal regret against arbitrary opponent behavior and, in ordinal potential games, almost-sure last-iterate convergence to the Nash set. Our results provide regret, dynamics, and equilibrium guarantees directly from preference feedback without reconstructing cardinal utilities.
ER -