The Normal Distributions Indistinguishability Spectrum and its Application to Privacy-Preserving Machine Learning
arXiv (Cornell University)(2023)
摘要
To achieve differential privacy (DP) one typically randomizes the output of
the underlying query. In big data analytics, one often uses randomized
sketching/aggregation algorithms to make processing high-dimensional data
tractable. Intuitively, such machine learning (ML) algorithms should provide
some inherent privacy, yet most if not all existing DP mechanisms do not
leverage this inherent randomness, resulting in potentially redundant noising.
The motivating question of our work is:
(How) can we improve the utility of DP mechanisms for randomized ML queries,
by leveraging the randomness of the query itself?
Towards a (positive) answer, we prove the Normal Distributions
Indistinguishability Spectrum Theorem (in short, NDIS Theorem), a theoretical
result with far-reaching practical implications. In a nutshell, NDIS is a
closed-form analytic computation for the
(ϵ,δ)-indistinguishability-spectrum (in short,
(ϵ,δ)-IS) of two arbitrary (multi-dimensional) normal
distributions X and Y, i.e., the optimal δ (for any given
ϵ) such that X and Y are (ϵ,δ)-close according to
the DP distance. The NDIS theorem (1) yields efficient estimators for the above
IS, and (2) allows us to analyze DP-mechanisms with normally-distributed
outputs, as well as more general mechanisms by leveraging their behavior on
large inputs.
We apply the NDIS theorem to derive DP mechanisms for queries with
normally-distributed outputs – i.e., Gaussian Random Projections (RP) – and
for more general queries – i.e., Ordinary Least Squares (OLS). Both RP and OLS
are highly relevant in data analytics. Our new DP mechanisms achieve superior
privacy/utility trade-offs by leveraging the randomness of the underlying
algorithms, and identifies, for the first time, the range of
(ϵ,δ) for which no additional noising is needed.
更多查看译文
关键词
random projections,ols,privacy-utility
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要