Adversarial Multiplicity: Cherry-Picked Models and the Limits of Disclosure
Abstract
Predictive multiplicity -- where similarly accurate models disagree on individual predictions -- can be leveraged to optimize secondary metrics (e.g., fairness) without loss in predictive performance. However, exploiting multiplicity opens the opportunity for motivated parties (or coding agents gaming their objectives) to steer predictions toward their own ends, such as in litigation or consumer disputes with automated decision-making tools. In these adversarial multiplicity settings, we distinguish soft manipulation, where parties optimize over innocuous model choices (e.g., hyperparameter tuning, feature engineering), from hard manipulation, where parties directly perturb model parameters. We find a sufficiently flexible model can be manipulated to reverse any individual prediction with minimal change in predictive performance while passing black-box audits, making hard manipulation viable in most regulatory settings. Soft manipulation has smaller effects but can pass white-box audits requiring full access to code and data. Even aggregate estimates can be manipulated: a model's estimated effect of climate change on wildfires can be fully reversed by hard edits and soft search. We find the Reconcile algorithm typically resolves most disagreements between models, but more often in favor of the manipulator when the opposing model is chosen adversarially compared to at random. We map which manipulations can plausibly pass existing regulations and disclosure policies, and recommend mitigations such as pre-registration, proof of model training, sensitivity analyses, multi-metric evaluations, and committing to versioned public models.
Related
- Limits of Amao multiplicities and Epsilon multiplicity
- Higher genus Cherry flows and full families of GIETs
- Quantile-Only Regression: Inference and Partial Identification under Quantile Disclosure
- Promoting Generalization for Exact Combinatorial Solvers via Adversarial Instance Augmentation
- Decoy and disclosure radii of invariant shape descriptors
BibTeX
@misc{chou2026adversarial,
title = {{Adversarial Multiplicity: Cherry-Picked Models and the Limits of Disclosure}},
author = {Cassandra C. Chou and Benjamin Q. Huynh and Allison Koenecke and Elizabeth T. Chin},
howpublished = {arXiv},
year = {2026},
month = nov,
doi = {10.48550/arxiv.2611.07378},
eprint = {2611.07378},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2611.07378},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Adversarial Multiplicity: Cherry-Picked Models and the Limits of Disclosure
AU - Cassandra C. Chou
AU - Benjamin Q. Huynh
AU - Allison Koenecke
AU - Elizabeth T. Chin
PY - 2026
DA - 2026/11/05
PB - arXiv
DO - 10.48550/arxiv.2611.07378
UR - https://arxiv.org/abs/2611.07378
AB - Predictive multiplicity -- where similarly accurate models disagree on individual predictions -- can be leveraged to optimize secondary metrics (e.g., fairness) without loss in predictive performance. However, exploiting multiplicity opens the opportunity for motivated parties (or coding agents gaming their objectives) to steer predictions toward their own ends, such as in litigation or consumer disputes with automated decision-making tools. In these adversarial multiplicity settings, we distinguish soft manipulation, where parties optimize over innocuous model choices (e.g., hyperparameter tuning, feature engineering), from hard manipulation, where parties directly perturb model parameters. We find a sufficiently flexible model can be manipulated to reverse any individual prediction with minimal change in predictive performance while passing black-box audits, making hard manipulation viable in most regulatory settings. Soft manipulation has smaller effects but can pass white-box audits requiring full access to code and data. Even aggregate estimates can be manipulated: a model's estimated effect of climate change on wildfires can be fully reversed by hard edits and soft search. We find the Reconcile algorithm typically resolves most disagreements between models, but more often in favor of the manipulator when the opposing model is chosen adversarially compared to at random. We map which manipulations can plausibly pass existing regulations and disclosure policies, and recommend mitigations such as pre-registration, proof of model training, sensitivity analyses, multi-metric evaluations, and committing to versioned public models.
ER -