Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments
Abstract
Interactive decision support systems often require repeated preference judgments, although identifying the best alternative may not justify eliciting all n(n−1)/2 pairwise comparisons. We propose ALEC (Adaptive Lexicographic Elicitation of Comparisons), which re-ranks the alternatives after each judgment and asks the first unobserved pair in the lexicographic order of the current ranking, using the Analytic Hierarchy Process as the concrete setting. Under a log-normal error model, directly comparing a contrast maximises the one-step reduction in its effective resistance. Posterior bounds on the probability of a wrong choice or of exceeding a relative-loss tolerance provide stopping rules valid under adaptive questioning, while a consistency-and-margin condition gives a deterministic certificate of agreement with the full-matrix winner. Simulations show that re-ranking is most useful in larger problems with limited budgets; ALEC outperforms Harker's original rule in most scenarios with n≥11, while under a common stopping rule it achieves higher hit-rate point estimates than the stronger Harker-log variant at a modest additional questioning cost. With a 5\% loss tolerance and a cap of 2n additional comparisons, the stopping rule saves \res{prot-s06-saveadd-min}--\res{prot-s06-saveadd-max}\% of them, while among runs stopped by the bound the observed rate of losses above tolerance remains at most \res{s06-errearly-max}\%, against a nominal 10\%. ALEC therefore provides an adaptive elicitation module for decision support systems that balances decision precision against user effort.
Related
- Adaptive Preference Elicitation in Preference Robust CPT-Based Shortfall
- Bayesian Deck-of-cards-based Ordinal Regression with Sequential Preference Elicitation
- Continuous Envy-Free Allocation via Reciprocal Preference Matrices: An Alternative to Sperner-Based Discretization
- DIAL: Position-Debiased LLM Judges with Adaptive Human Preference Calibration
- BayesJudge: Uncertainty-Aware Bayesian Meta-Evaluation of Human and LLM Judgments
BibTeX
@misc{calvo2026adaptive,
title = {{Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments}},
author = {Luis Ángel Calvo and Jiri Mazurek},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.08123},
eprint = {2611.08123},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.08123},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Adaptive pairwise preference elicitation for decision support: identifying the best alternative with fewer judgments
AU - Luis Ángel Calvo
AU - Jiri Mazurek
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.08123
UR - https://arxiv.org/abs/2611.08123
AB - Interactive decision support systems often require repeated preference judgments, although identifying the best alternative may not justify eliciting all n(n−1)/2 pairwise comparisons. We propose ALEC (Adaptive Lexicographic Elicitation of Comparisons), which re-ranks the alternatives after each judgment and asks the first unobserved pair in the lexicographic order of the current ranking, using the Analytic Hierarchy Process as the concrete setting. Under a log-normal error model, directly comparing a contrast maximises the one-step reduction in its effective resistance. Posterior bounds on the probability of a wrong choice or of exceeding a relative-loss tolerance provide stopping rules valid under adaptive questioning, while a consistency-and-margin condition gives a deterministic certificate of agreement with the full-matrix winner. Simulations show that re-ranking is most useful in larger problems with limited budgets; ALEC outperforms Harker's original rule in most scenarios with n≥11, while under a common stopping rule it achieves higher hit-rate point estimates than the stronger Harker-log variant at a modest additional questioning cost. With a 5\% loss tolerance and a cap of 2n additional comparisons, the stopping rule saves \res{prot-s06-saveadd-min}--\res{prot-s06-saveadd-max}\% of them, while among runs stopped by the bound the observed rate of losses above tolerance remains at most \res{s06-errearly-max}\%, against a nominal 10\%. ALEC therefore provides an adaptive elicitation module for decision support systems that balances decision precision against user effort.
ER -