Abstract
We consider three different variants of differential privacy (DP), namely approximate DP, Rényi DP (RDP), and hypothesis test DP. In the first part, we develop a machinery for optimally relating approximate DP to RDP based on the joint range of two f-divergences that underlie the approximate DP and RDP. In particular, this enables us to derive the optimal approximate DP parameters of a mechanism that satisfies a given level of RDP. As an application, we apply our result to the moments accountant framework for characterizing privacy guarantees of noisy stochastic gradient descent (SGD). When compared to the state-of-the-art, our bounds may lead to about 100 more stochastic gradient descent iterations for training deep learning models for the same privacy budget. In the second part, we establish a relationship between RDP and hypothesis test DP which allows us to translate the RDP constraint into a tradeoff between type I and type II error probabilities of a certain binary hypothesis test. We then demonstrate that for noisy SGD our result leads to tighter privacy guarantees compared to the recently proposed f-DP framework for some range of parameters.
Original language | English (US) |
---|---|
Article number | 9336023 |
Pages (from-to) | 208-222 |
Number of pages | 15 |
Journal | IEEE Journal on Selected Areas in Information Theory |
Volume | 2 |
Issue number | 1 |
DOIs | |
State | Published - Mar 2021 |
Keywords
- Binary hypothesis testing
- Differential privacy
- F-divergences
- Moments accountant
- Rényi divergence
- Stochastic gradient descent
ASJC Scopus subject areas
- Computer Networks and Communications
- Media Technology
- Artificial Intelligence
- Applied Mathematics