Conformal ordering: Distribution-free predictive inference for orders and rank
Abstract
We study distribution-free predictive inference for the ordering and ranking of multiple unobserved outcomes. Although numerous methods, such as conformal prediction, have been developed for prediction and predictive inference of unobserved outcomes, extending them to provide inference for the ordering of multiple outcomes with an interpretable guarantee is not straightforward. To this end, we develop a framework and methodology for uncertainty quantification in predictive ordering. Our approach begins by reformulating the problem of determining an ordering among multiple objects as a graph inference problem. Our method adopts a multiple-testing-inspired approach and constructs an uncertainty graph, where each edge represents a confidently inferred pairwise ordering. We develop a distribution-free procedure for constructing a consistent uncertainty graph while controlling either the family-wise error rate or the false discovery rate. Moreover, we show that the resulting graphs naturally yield risk-controlling prediction intervals for the ranks of the outcomes. Empirical results demonstrate that the proposed procedure achieves the target guarantees tightly while maintaining reasonable power and prediction-interval widths.
Related
- SCOPE Before You Sample: Conformal Scores Guide Posterior Exploration for Fast and Calibrated Bayesian Predictive Inference
- Impossibility of Distribution-Free Predictive Inference for Individual Treatment Effects
- Distribution-free predictive inference with instrumental variables
- A Shape-Adaptive Gamma-Mixture Power Shanker Distribution: Boundary-Aware Inference, Censored Regression, and Predictive Validation
- Adaptive Conformal Inference through the Lens of Blackwell Approachability
BibTeX
@misc{lee2026conformal,
title = {{Conformal ordering: Distribution-free predictive inference for orders and rank}},
author = {Yonghoon Lee and Zhimei Ren},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.40340},
eprint = {2610.40340},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.40340},
}
RIS
TY - UNPB
TI - Conformal ordering: Distribution-free predictive inference for orders and rank
AU - Yonghoon Lee
AU - Zhimei Ren
PY - 2026
DA - 2026/10/29
PB - arXiv
DO - 10.48550/arxiv.2610.40340
UR - https://arxiv.org/abs/2610.40340
AB - We study distribution-free predictive inference for the ordering and ranking of multiple unobserved outcomes. Although numerous methods, such as conformal prediction, have been developed for prediction and predictive inference of unobserved outcomes, extending them to provide inference for the ordering of multiple outcomes with an interpretable guarantee is not straightforward. To this end, we develop a framework and methodology for uncertainty quantification in predictive ordering. Our approach begins by reformulating the problem of determining an ordering among multiple objects as a graph inference problem. Our method adopts a multiple-testing-inspired approach and constructs an uncertainty graph, where each edge represents a confidently inferred pairwise ordering. We develop a distribution-free procedure for constructing a consistent uncertainty graph while controlling either the family-wise error rate or the false discovery rate. Moreover, we show that the resulting graphs naturally yield risk-controlling prediction intervals for the ranks of the outcomes. Empirical results demonstrate that the proposed procedure achieves the target guarantees tightly while maintaining reasonable power and prediction-interval widths.
ER -