Effect-Size Estimation and Confounding-Aware Permutations for PERMANOVA
Abstract
Motivation: PERMANOVA is widely used to test associations between covariates and high-dimensional data with arbitrary distributions, such as ecology, metabolomics, or microbial community profiling. In these contexts, its effect size measure (R2) is generally used to indicate how much the high-dimensional measurement covaries with respect to a separate, univariate measure. However, as we show, this conventional effect size is inflated under the null, resulting in values that do not accurately reflect the conventional expectation of "variance explained." Furthermore, standard permutation schemes yield inflated false positives in the presence of non-independent samples, such as repeated measure designs. Results: We introduce confounding-aware permutation schemes for both invariant and variant covariates. We additionally develop a causally-grounded, distance-based effect-size estimator derived from the Hájek average treatment effect estimator, providing interpretable effect sizes in the units of the chosen dissimilarity metric. Simulations using microbiome profiles demonstrate that these approaches control false positives while maintaining power, outperforming standard PERMANOVA implementations. Availability and Implementation: An R implementation is freely available in the permaBrioche module of the bioBakery ecosystem https://huttenhower.sph.harvard.edu/permabrioche. Contact: [email protected]
Related
- Quadratic-size Random Permutations are Universal
- Proximal Balancing for Causal Effect Estimation under Unmeasured Confounding
- Nesting or resolution? Facility proxies for variance estimation versus confounding adjustment in claims data without facility identifiers
- Mixture-based Nonparametric Estimation of Spatial Covariance Functions with Applications to HIV Key Population Size Estimation across Sub-Saharan Africa
- Bayesian Sample Size Determination: Sampling Distribution Estimation or Exploration?
BibTeX
@misc{manage2026effectsize,
title = {{Effect-Size Estimation and Confounding-Aware Permutations for PERMANOVA}},
author = {Sithija Manage and Emma Accorsi and Jacob Nearing and Siyuan Ma and Eric Franzosa and Curtis Huttenhower},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.36519},
eprint = {2610.36519},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.36519},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Effect-Size Estimation and Confounding-Aware Permutations for PERMANOVA
AU - Sithija Manage
AU - Emma Accorsi
AU - Jacob Nearing
AU - Siyuan Ma
AU - Eric Franzosa
AU - Curtis Huttenhower
PY - 2026
DA - 2026/10/27
PB - arXiv
DO - 10.48550/arxiv.2610.36519
UR - https://arxiv.org/abs/2610.36519
AB - Motivation: PERMANOVA is widely used to test associations between covariates and high-dimensional data with arbitrary distributions, such as ecology, metabolomics, or microbial community profiling. In these contexts, its effect size measure (R2) is generally used to indicate how much the high-dimensional measurement covaries with respect to a separate, univariate measure. However, as we show, this conventional effect size is inflated under the null, resulting in values that do not accurately reflect the conventional expectation of "variance explained." Furthermore, standard permutation schemes yield inflated false positives in the presence of non-independent samples, such as repeated measure designs. Results: We introduce confounding-aware permutation schemes for both invariant and variant covariates. We additionally develop a causally-grounded, distance-based effect-size estimator derived from the Hájek average treatment effect estimator, providing interpretable effect sizes in the units of the chosen dissimilarity metric. Simulations using microbiome profiles demonstrate that these approaches control false positives while maintaining power, outperforming standard PERMANOVA implementations. Availability and Implementation: An R implementation is freely available in the permaBrioche module of the bioBakery ecosystem https://huttenhower.sph.harvard.edu/permabrioche. Contact: [email protected]
ER -