CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds
Abstract
Public benchmark scores may reflect skill, prior exposure to the questions, or both, and for most models the training data are unknown. We present CleanScore, a black-box audit using scored outputs only. Each benchmark question becomes a parent item with one public form and two independently written fresh forms preserving its numbers, facts and answer. The audit reports an interval for the public-form advantage rather than a verdict, and a private negative-control bank with an explicit transport radius separates exposure from ordinary form mismatch. A registered controlled-exposure experiment detects planted exposure and stays quiet under fresh-form exposure. A registered audit of five open models on 200 GSM8K and 200 ARC-Challenge items finds no exposure-consistent advantage, bounding surface-form inflation below five points. Registered positive controls then bound what such a null can mean. Leaking an item raises accuracy on paraphrases the model never saw almost as much as on the leaked wording, leaving 52% to 110% of the effect invisible to a paraphrase audit. On ARC a planted 49-point advantage shows an observable gap of -0.020, and about 20 points survive rewriting stem and options, across four training seeds. A surface-form null bounds far less than the phrase contamination audit implies.
Related
- Negative controls for bias detection within exposure-anchored self-controlled designs: applications and considerations
- TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
- Graphene-Assisted Electrostatic Screening of Resonant Transport in MR-DWELL Dosimeters with Tunable Sensitivity--Radiation Tolerance Trade-off: A Reduced-Order Benchmark and Self-Consistent Perspective
- Reference-based sensitivity analysis for repeated count outcomes using negative binomial margins and a Gaussian copula
- Risk-Limiting Audits for Parliamentary Majorities
BibTeX
@misc{opoku2026cleanscore,
title = {{CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds}},
author = {Jeffery Opoku and David Banahene},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.22183},
eprint = {2609.22183},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.22183},
}
RIS
TY - UNPB
TI - CleanScore: Black-Box Benchmark Audits with Negative Controls and Sensitivity Bounds
AU - Jeffery Opoku
AU - David Banahene
PY - 2026
DA - 2026/09/01
PB - arXiv
DO - 10.48550/arxiv.2609.22183
UR - https://arxiv.org/abs/2609.22183
AB - Public benchmark scores may reflect skill, prior exposure to the questions, or both, and for most models the training data are unknown. We present CleanScore, a black-box audit using scored outputs only. Each benchmark question becomes a parent item with one public form and two independently written fresh forms preserving its numbers, facts and answer. The audit reports an interval for the public-form advantage rather than a verdict, and a private negative-control bank with an explicit transport radius separates exposure from ordinary form mismatch. A registered controlled-exposure experiment detects planted exposure and stays quiet under fresh-form exposure. A registered audit of five open models on 200 GSM8K and 200 ARC-Challenge items finds no exposure-consistent advantage, bounding surface-form inflation below five points. Registered positive controls then bound what such a null can mean. Leaking an item raises accuracy on paraphrases the model never saw almost as much as on the leaked wording, leaving 52% to 110% of the effect invisible to a paraphrase audit. On ARC a planted 49-point advantage shows an observable gap of -0.020, and about 20 points survive rewriting stem and options, across four training seeds. A surface-form null bounds far less than the phrase contamination audit implies.
ER -