Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries
Abstract
Finite safety assessments cannot exclude an adversary that preserves unattacked behavior until certification and activates afterward. We characterize this limitation for embodied and tool-use AI. An adversary that matches the complete observation law until observable, irrevocable certification and retains intervention access afterward attains v𝔼[(n−S0)+] expected post-certification events over n trials, where v is the active event probability and S0 the unattacked certification time. The result covers adaptive and privately randomized evaluators. Switching at certification is exactly optimal for a fixed likelihood-ratio test under memoryless, event-increasing interventions without a binding intervention budget. A positive clean-certification requirement forces linear worst-case expected excess when intervention raises event rates over an authorized deployment horizon, even if certification is deferred to the deadline. In simulated RoCo collaboration, switching yields commanded-separation violations in 34 of 36 post-certification episodes. Preregistered randomized AgentDojo trials estimate 1.98 additional security violations per four-task continuation (95% interval [1.67, 2.29]). An adversarial language-model controller infers hidden certification from noisy evaluation receipts and times intervention accordingly. With permanent revocation, a mixture likelihood-ratio monitor bounds cumulative excess above a prespecified reference rate at square-root order, up to logarithmic factors, on every sample path. A single fixed alternative can permit linear expected excess under stationary attacks. Against a controller observing the monitor's rule and statistic, live mixture monitoring reduces violations from 12.00 to 3.81 per 24 assigned task opportunities. Revocation also reduces tasks executed.
Related
- Statistical-computational trade-offs for recursive adaptive partitioning estimators
- Information-Geometric Grammar Principle: A Statistical-Geometric Framework for Adaptive Systems
- Constrained Gradient Equilibrium: Approachability under Locally Constrained Adversaries
- Authentication over Arbitrarily Varying Channels with Causal Adversaries
- Le système intermédiaire d'assurance
BibTeX
@misc{singh2026limits,
title = {{Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries}},
author = {Inderjeet Singh},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.18948},
eprint = {2610.18948},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.18948},
}
RIS
TY - UNPB
TI - Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries
AU - Inderjeet Singh
PY - 2026
DA - 2026/10/13
PB - arXiv
DO - 10.48550/arxiv.2610.18948
UR - https://arxiv.org/abs/2610.18948
AB - Finite safety assessments cannot exclude an adversary that preserves unattacked behavior until certification and activates afterward. We characterize this limitation for embodied and tool-use AI. An adversary that matches the complete observation law until observable, irrevocable certification and retains intervention access afterward attains v𝔼[(n−S_0)_+] expected post-certification events over n trials, where v is the active event probability and S_0 the unattacked certification time. The result covers adaptive and privately randomized evaluators. Switching at certification is exactly optimal for a fixed likelihood-ratio test under memoryless, event-increasing interventions without a binding intervention budget. A positive clean-certification requirement forces linear worst-case expected excess when intervention raises event rates over an authorized deployment horizon, even if certification is deferred to the deadline. In simulated RoCo collaboration, switching yields commanded-separation violations in 34 of 36 post-certification episodes. Preregistered randomized AgentDojo trials estimate 1.98 additional security violations per four-task continuation (95% interval [1.67, 2.29]). An adversarial language-model controller infers hidden certification from noisy evaluation receipts and times intervention accordingly. With permanent revocation, a mixture likelihood-ratio monitor bounds cumulative excess above a prespecified reference rate at square-root order, up to logarithmic factors, on every sample path. A single fixed alternative can permit linear expected excess under stationary attacks. Against a controller observing the monitor's rule and statistic, live mixture monitoring reduces violations from 12.00 to 3.81 per 24 assigned task opportunities. Revocation also reduces tasks executed.
ER -