Probabilistic Adversarial Training
Abstract
Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution pdis and a victim-classifier-induced distribution pvic, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that KL(pdis|pvic)−log Zvic is a lower bound on probabilistic robustness (PR), where Zvic denotes the normalizing constant of pvic. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.
Related
- On the tradeoff between robustness and geometric accuracy in adversarial training
- Statistical Inference for Adversarial Training: Central Limit Theorems via Optimal Transport
- Distributionally robust linear regression through the lens of adversarial training
- Brenier Meets Adversarial Training: Optimal Transport Geometry for Robust Learning
- Adversarial Training with Large Step Sizes: Implicit Bias, Optimization, and Sharpness
BibTeX
@misc{zhang2026probabilistic,
title = {{Probabilistic Adversarial Training}},
author = {Andi Zhang and Xingyu Zhao and Siddartha Khastgir},
howpublished = {arXiv},
year = {2026},
month = sep,
doi = {10.48550/arxiv.2609.39798},
eprint = {2609.39798},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.39798},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Probabilistic Adversarial Training
AU - Andi Zhang
AU - Xingyu Zhao
AU - Siddartha Khastgir
PY - 2026
DA - 2026/09/30
PB - arXiv
DO - 10.48550/arxiv.2609.39798
UR - https://arxiv.org/abs/2609.39798
AB - Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution p_dis and a victim-classifier-induced distribution p_vic, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that KL(p_dis|p_vic)−log Z_vic is a lower bound on probabilistic robustness (PR), where Z_vic denotes the normalizing constant of p_vic. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.
ER -