Can Rényi Accountants Beat Lossless Accountants for DP-SGD?
Abstract
Recent frameworks in differential privacy (DP), such as privacy loss distributions and f-DP, provide lossless representations of the privacy leakage of mechanisms. These approaches are often considered to yield tighter (ε, δ)-DP guarantees than those obtained from relaxed notions such as Rényi DP. But is that always the case? We study this question for differentially private stochastic gradient descent (DP-SGD) with fixed-size batches under replace adjacency. Our key observation is that in this setting, privacy accounting based on Rényi DP can use a pair of worst-case distributions (i.e., a dominating pair) that differs from the one used in the lossless frameworks. In particular, we characterize the worst-case distributions as two-dimensional Gaussian mixtures and derive an improved Rényi DP bound for DP-SGD. Numerical and asymptotic comparisons show that across various hyperparameter settings, a Rényi DP accountant based on the new bound can yield better privacy guarantees than those obtained from the lossless frameworks.
Related
- Ornstein-Uhlenbeck Is Hard to Beat, Yet Superlinear Drift Ships Lower Transport Costs
- Asymptotically Lossless Exact and Pivotal E-Values
- From Geometry to Generalization: Why Row Normalization Can Beat Adam and Muon
- Rates and Nearly-Lossless State over a Multi-Access Channel with Causal State Information
- Lossless Source Coding with Marginal Distribution Constraint
BibTeX
@misc{bok2026rényi,
title = {{Can Rényi Accountants Beat Lossless Accountants for DP-SGD?}},
author = {Jinho Bok},
howpublished = {arXiv},
year = {2026},
month = oct,
doi = {10.48550/arxiv.2610.17120},
eprint = {2610.17120},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2610.17120},
}
RIS
TY - UNPB
TI - Can Rényi Accountants Beat Lossless Accountants for DP-SGD?
AU - Jinho Bok
PY - 2026
DA - 2026/10/12
PB - arXiv
DO - 10.48550/arxiv.2610.17120
UR - https://arxiv.org/abs/2610.17120
AB - Recent frameworks in differential privacy (DP), such as privacy loss distributions and f-DP, provide lossless representations of the privacy leakage of mechanisms. These approaches are often considered to yield tighter (ε, δ)-DP guarantees than those obtained from relaxed notions such as Rényi DP. But is that always the case? We study this question for differentially private stochastic gradient descent (DP-SGD) with fixed-size batches under replace adjacency. Our key observation is that in this setting, privacy accounting based on Rényi DP can use a pair of worst-case distributions (i.e., a dominating pair) that differs from the one used in the lossless frameworks. In particular, we characterize the worst-case distributions as two-dimensional Gaussian mixtures and derive an improved Rényi DP bound for DP-SGD. Numerical and asymptotic comparisons show that across various hyperparameter settings, a Rényi DP accountant based on the new bound can yield better privacy guarantees than those obtained from the lossless frameworks.
ER -