Improved Communication-Privacy Trade-offs in L_2 Mean Estimation under Streaming Differential Privacy
arxiv(2024)
摘要
We study L_2 mean estimation under central differential privacy and
communication constraints, and address two key challenges: firstly, existing
mean estimation schemes that simultaneously handle both constraints are usually
optimized for L_∞ geometry and rely on random rotation or Kashin's
representation to adapt to L_2 geometry, resulting in suboptimal leading
constants in mean square errors (MSEs); secondly, schemes achieving
order-optimal communication-privacy trade-offs do not extend seamlessly to
streaming differential privacy (DP) settings (e.g., tree aggregation or matrix
factorization), rendering them incompatible with DP-FTRL type optimizers.
In this work, we tackle these issues by introducing a novel privacy
accounting method for the sparsified Gaussian mechanism that incorporates the
randomness inherent in sparsification into the DP noise. Unlike previous
approaches, our accounting algorithm directly operates in L_2 geometry,
yielding MSEs that fast converge to those of the uncompressed Gaussian
mechanism. Additionally, we extend the sparsification scheme to the matrix
factorization framework under streaming DP and provide a precise accountant
tailored for DP-FTRL type optimizers. Empirically, our method demonstrates at
least a 100x improvement of compression for DP-SGD across various FL tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要