Power-Optimal Risk Control via Standardized Margin Calibration
Abstract
Machine learning can screen far more candidates than can be experimentally validated, creating a need for selection procedures that identify promising candidates while controlling risk before costly experimental validation. We propose Uncertainty-Normalized Empirical Calibration (UNEC), a p-value-free framework that ranks candidates by uncertainty-normalized predictive margins and directly calibrates a selection cutoff using held-out losses. UNEC supports flexible prediction models, general bounded losses, and diverse selection targets. We study marginal discovery risk (MDR) and selective discovery risk (SDR), with SDR reducing to the false discovery rate under binary loss. We establish finite-sample MDR control for an adjusted rule and asymptotic SDR control with explicit limiting risk and power. Under a location--scale model with a common one-sided binary target, we further show that UNEC asymptotically attains the oracle power benchmark among observation-wise selection rules. Simulations and molecular screening applications in solubility, toxicity, and lipophilicity demonstrate favorable risk--power trade-offs and substantial computational gains across binary and graded-loss settings.
Related
- Bias-Aware Adjoint Calibration and Optimal Control of Stochastic Multiscale Bioprocess Digital Twins
- A New Perspective on Optimal Power Flow: From Control Theory to Numerical Optimization
- BOP2-ENR: a Bayesian optimal phase II design for adaptive enrichment with family-wise error rate control and a bound on joint-claim power
- Optimal VC Dimension of Contrastive Learning with Margin
- Finite-Sample Calibration and Power Benchmarks in Two-Sample Random-Integration Tests
BibTeX
@misc{wang2026poweroptimal,
title = {{Power-Optimal Risk Control via Standardized Margin Calibration}},
author = {Ruinan Wang and Tian Bai and Yueming Shen and Xiang Yu and Ying Yang and Archer Yi Yang},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.00480},
eprint = {2611.00480},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.00480},
note = {CC BY-NC-ND 4.0},
}
RIS
TY - UNPB
TI - Power-Optimal Risk Control via Standardized Margin Calibration
AU - Ruinan Wang
AU - Tian Bai
AU - Yueming Shen
AU - Xiang Yu
AU - Ying Yang
AU - Archer Yi Yang
PY - 2026
DA - 2026/11/01
PB - arXiv
DO - 10.48550/arxiv.2611.00480
UR - https://arxiv.org/abs/2611.00480
AB - Machine learning can screen far more candidates than can be experimentally validated, creating a need for selection procedures that identify promising candidates while controlling risk before costly experimental validation. We propose Uncertainty-Normalized Empirical Calibration (UNEC), a p-value-free framework that ranks candidates by uncertainty-normalized predictive margins and directly calibrates a selection cutoff using held-out losses. UNEC supports flexible prediction models, general bounded losses, and diverse selection targets. We study marginal discovery risk (MDR) and selective discovery risk (SDR), with SDR reducing to the false discovery rate under binary loss. We establish finite-sample MDR control for an adjusted rule and asymptotic SDR control with explicit limiting risk and power. Under a location--scale model with a common one-sided binary target, we further show that UNEC asymptotically attains the oracle power benchmark among observation-wise selection rules. Simulations and molecular screening applications in solubility, toxicity, and lipophilicity demonstrate favorable risk--power trade-offs and substantial computational gains across binary and graded-loss settings.
ER -