Anytime-valid detection of LLM weight exfiltration
Abstract
A compromised LLM inference server can leak model weights by encoding payload bits in otherwise plausible token choices. A replay of the same prompt in a trusted server can expose such deviations, but benign numerical nondeterminism also causes token mismatches. Patient attackers can therefore hide within normal variation unless evidence is combined across responses. We introduce a prompt-level e-process that calibrates whole-response mismatch events on trusted benign traffic and accumulates evidence sequentially while, under a calibration-transfer assumption, controlling the probability of any false alarm over an unbounded monitoring horizon. We evaluate it on four models against a seed-blind attack and a stronger seed-aware attack that hides payload bits only in near-ties to remain stealthy, analyzing the channel capacity vs detectability trade-off. Compared with a hard per-token alarm, the e-process combines weak evidence across responses while providing explicit anytime false-alarm control.
Related
- Anytime-Valid Distribution Shift Detection via Predictive Rank Martingales
- Anytime-Valid Inference in Linear Models with Applications to Regression-Adjusted Causal Inference
- Asymptotic Anytime-Valid Quantile Inference under Local Differential Privacy
- Anytime Valid Confidence Bands for Distribution Functions
- Boot-AVT: Asymptotically Anytime-Valid Testing with Degenerate U-Statistics via Bootstrap
BibTeX
@misc{ortegafernandez2026anytimevalid,
title = {{Anytime-valid detection of LLM weight exfiltration}},
author = {Ines Ortega-Fernandez and Mateusz Kowalczyk and Keri Warr},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.11843},
eprint = {2610.11843},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.11843},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Anytime-valid detection of LLM weight exfiltration
AU - Ines Ortega-Fernandez
AU - Mateusz Kowalczyk
AU - Keri Warr
PY - 2026
DA - 2026/10/08
PB - arXiv
DO - 10.48550/arxiv.2610.11843
UR - https://arxiv.org/abs/2610.11843
AB - A compromised LLM inference server can leak model weights by encoding payload bits in otherwise plausible token choices. A replay of the same prompt in a trusted server can expose such deviations, but benign numerical nondeterminism also causes token mismatches. Patient attackers can therefore hide within normal variation unless evidence is combined across responses. We introduce a prompt-level e-process that calibrates whole-response mismatch events on trusted benign traffic and accumulates evidence sequentially while, under a calibration-transfer assumption, controlling the probability of any false alarm over an unbounded monitoring horizon. We evaluate it on four models against a seed-blind attack and a stronger seed-aware attack that hides payload bits only in near-ties to remain stealthy, analyzing the channel capacity vs detectability trade-off. Compared with a hard per-token alarm, the e-process combines weak evidence across responses while providing explicit anytime false-alarm control.
ER -