Privacy-Preserving Convolutional Neural Networks Using Homomorphic Encryption

Convolutional neural networks (CNNs) are most commonly used for handling complex visual tasks. Due to the significant effort in training accurate machine learning models, training and providing them to clients for inference in the cloud is becoming more popular. However, such models might be trained on sensitive user data, leading to the need for privacy-protecting measures. While traditional cryptographic techniques do not allow operations in the encrypted domain, Homomorphic Encryption (HE) schemes enable us to work on encrypted data directly. Combining the concepts of CNNs and HE, we give a detailed overview of the steps involved in creating privacy-preserving neural networks to give an indication of the real-world applicability and the scalability of such an approach. To this extent, we implemented a homomorphically encrypted network that can be used for face recognition. Our results indicate that while we can achieve the same accuracy as a standard neural network, running CNNs on homomorphically encrypted inputs comes at a significant overhead that grows with the network size.