How Wrong Can a Rank-Based Sample-Size Calculation Be? A Sharp Bound of 16/9 for Ordinal Outcomes
Abstract
Rank-based tests need no distributional assumptions to be valid, but the sample size they require does depend on the shapes of the two outcome distributions, which are unknown at the design stage. Standard practice substitutes a variance calibrated under the null. We ask how wrong that substitution can be. Writing Π for the ratio of the true asymptotic variance of the estimated relative effect to the substituted one, we prove that under balanced allocation Π≤ 16θ(1−θ)/{2+θ(1−θ)} ≤ 16/9 for all ordinal distributions in any number of categories: within the first-order asymptotic calculation the required sample size can exceed the calculated one by at most 77.8%, and an explicit two-point family attains the bound at every effect size. The effect-specific envelope, not the constant, is the operative quantity for a design: the ceiling falls to $1.63$ at θ=0.65 and $1.37$ at θ=0.75, so a trial powered for a larger effect is correspondingly less exposed. The proof is elementary. In five extracted trial comparisons and 4,711 generated alternatives the realized cost stays within a few percent, because the extremal configuration is one that shift-type treatment mechanisms do not produce. These facts are complementary, and fix how the bound should be used: it is a design sensitivity certificate, not an inflation factor. A protocol can report the conventional sample size alongside the largest requirement consistent with any ordinal configuration.
Related
- On Power and Sample Size Calculation for Bayes Factors: A Tutorial on the BayesPower Package
- Orthogonal Policy Learning with Ordinal Outcomes
- Sample size calculation comparing two survival curves in the presence of nonproportional hazards utilizing the concept of Weighted Relative Time using two different Weibull distributions
- Is the treatment beneficial? Causal Inference with Ordinal Outcomes: A Bayesian Solution
- Doubly Robust Estimation and Inference for Longitudinal Win Probabilities with Ordinal Outcomes in Observational Studies
BibTeX
@misc{phaibulpanich2026wrong,
title = {{How Wrong Can a Rank-Based Sample-Size Calculation Be? A Sharp Bound of 16/9 for Ordinal Outcomes}},
author = {Akarin Phaibulpanich},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.06575},
eprint = {2609.06575},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.06575},
}
RIS
TY - UNPB
TI - How Wrong Can a Rank-Based Sample-Size Calculation Be? A Sharp Bound of 16/9 for Ordinal Outcomes
AU - Akarin Phaibulpanich
PY - 2026
DA - 2026/09/06
PB - arXiv
DO - 10.48550/arxiv.2609.06575
UR - https://arxiv.org/abs/2609.06575
AB - Rank-based tests need no distributional assumptions to be valid, but the sample size they require does depend on the shapes of the two outcome distributions, which are unknown at the design stage. Standard practice substitutes a variance calibrated under the null. We ask how wrong that substitution can be. Writing Π for the ratio of the true asymptotic variance of the estimated relative effect to the substituted one, we prove that under balanced allocation Π≤ 16θ(1−θ)/{2+θ(1−θ)} ≤ 16/9 for all ordinal distributions in any number of categories: within the first-order asymptotic calculation the required sample size can exceed the calculated one by at most 77.8%, and an explicit two-point family attains the bound at every effect size. The effect-specific envelope, not the constant, is the operative quantity for a design: the ceiling falls to $1.63$ at θ=0.65 and $1.37$ at θ=0.75, so a trial powered for a larger effect is correspondingly less exposed. The proof is elementary. In five extracted trial comparisons and 4,711 generated alternatives the realized cost stays within a few percent, because the extremal configuration is one that shift-type treatment mechanisms do not produce. These facts are complementary, and fix how the bound should be used: it is a design sensitivity certificate, not an inflation factor. A protocol can report the conventional sample size alongside the largest requirement consistent with any ordinal configuration.
ER -