Selecting Informative Conformal Prediction Sets with an Optimized FCR-Controlled Approach
Abstract
Conformal methods provide prediction sets for outcomes with confidence guarantees. We study their use in a selective inference setting, where inference is performed only when the prediction set is informative. The analyst may consider as informative, for example, cases with prediction sets that are sufficiently small, exclude null values, or satisfy other appropriate monotone constraints. Because inference is typically restricted to informative cases in practical applications, accounting for the resulting selection bias is crucial to maintaining false coverage rate (FCR) control. A general framework for constructing such informative conformal prediction sets while controlling the FCR on the selected sample was suggested in Gazin et al. (2025). In this work we focus on oracle-guided procedures. We derive the optimal decision policy under a suitable power objective in the oracle setting where the probability of belonging to each prediction set can be computed. In practice, of course, only estimated probabilities are available. We therefore introduce calibration procedures that adjust the oracle policy to maintain finite-sample FCR control. We show that this approach can achieve substantially higher power than available alternatives. We demonstrate the effectiveness of our new methods for classification outcomes on both real and simulated data.
Related
- Conformal Risk-Averse Decision Making with Optimized Certainty Equivalent Risk Control
- Generalized Hyperrectangular Prediction Sets in Multivariate Conformal Prediction
- Conformal Prediction under Partial Verification
- Multi-Variable Conformal Prediction: Optimizing Prediction Sets using Multiple Calibration Variables
- Conformal Prediction and Conditional Coverage for Tabular Foundation Models
BibTeX
@misc{solomon2026selecting,
title = {{Selecting Informative Conformal Prediction Sets with an Optimized FCR-Controlled Approach}},
author = {Israela Solomon and Etienne Roquain and Saharon Rosset and Ruth Heller},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2605.22004},
eprint = {2605.22004},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2605.22004},
}
RIS
TY - UNPB
TI - Selecting Informative Conformal Prediction Sets with an Optimized FCR-Controlled Approach
AU - Israela Solomon
AU - Etienne Roquain
AU - Saharon Rosset
AU - Ruth Heller
PY - 2026
DA - 2026/10/05
PB - arXiv
DO - 10.48550/arxiv.2605.22004
UR - https://arxiv.org/abs/2605.22004
AB - Conformal methods provide prediction sets for outcomes with confidence guarantees. We study their use in a selective inference setting, where inference is performed only when the prediction set is informative. The analyst may consider as informative, for example, cases with prediction sets that are sufficiently small, exclude null values, or satisfy other appropriate monotone constraints. Because inference is typically restricted to informative cases in practical applications, accounting for the resulting selection bias is crucial to maintaining false coverage rate (FCR) control. A general framework for constructing such informative conformal prediction sets while controlling the FCR on the selected sample was suggested in Gazin et al. (2025). In this work we focus on oracle-guided procedures. We derive the optimal decision policy under a suitable power objective in the oracle setting where the probability of belonging to each prediction set can be computed. In practice, of course, only estimated probabilities are available. We therefore introduce calibration procedures that adjust the oracle policy to maintain finite-sample FCR control. We show that this approach can achieve substantially higher power than available alternatives. We demonstrate the effectiveness of our new methods for classification outcomes on both real and simulated data.
ER -