Exploring privacy measurement in federated learning

Federated learning (FL) is a collaborative artificial intelligence (AI) approach that enables distributed training of AI models without data sharing, thereby promoting privacy by design. However, it is essential to acknowledge that FL only offers a partial solution to safeguard the confidentiality of AI and machine learning (ML) models. Unfortunately, many studies fail to report the results of privacy measurement when applying FL, mainly due to assumptions that privacy is implicitly achieved as FL is a privacy-by-design approach. This trend can also be attributed to the complexity of understanding privacy measurement metrics and methods. This paper presents a survey of privacy measurement in FL, aimed at evaluating its effectiveness in protecting the privacy of sensitive data during the training of AI and ML models. While FL is a promising approach for preserving privacy during model training, ensuring privacy is genuinely achieved in practice is crucial. By evaluating privacy measurement metrics and methods in FL, we can identify the gaps in existing approaches and propose new techniques to enhance FL’s privacy. A comprehensive study investigating “privacy measurement and metrics” in FL is therefore required to support the field’s growth. Our survey provides a critical analysis of the current state of privacy measurement in FL, identifies gaps in existing research, and offers insights into potential research directions. Moreover, this paper presents a case study that evaluates the effectiveness of various privacy techniques in a specific FL scenario. This case study serves as tangible evidence of the real-world implications of privacy measurements, providing insightful and practical guidelines for researchers and practitioners to optimize privacy preservation while balancing other crucial factors such as communication overhead and accuracy. Finally, our paper outlines a future roadmap for advancing privacy in FL, combining traditional techniques with innovative technologies such as quantum computing and Trusted Execution Environments to fortify data protection.