A new perceptual hashing method for verification and identity classification of occluded faces

AbstractAbstractRecently, research communities on Computer Vision and biometrics have shown a lot of interest in face verification and classification methods. Fighting against Child Sexual Exploitation Material (CSEM) is one of the applications that might benefit most from these advances. In CSEM, discriminative parts of the face, i.e. mostly the eyes, are often occluded to make the victim identification more difficult. Most of the current face recognition methods are not able to handle such kind of occlusions. To overcome this problem, we present One-Shot Frequency Dominant Neighborhood Structure (OSF-DNS), a new perceptual hashing method that shows advantages on two scenarios: (a) occluded face verification, i.e., matching occluded faces with their non-occluded versions, and (b) face classification, i.e., getting the identity of an occluded face by means of a classifier trained with the non-occluded faces using the perceptual hash codes as feature vectors. We have compared the face verification performance of OSF-DNS with three perceptual hashing methods and with the features obtained from five deep learning techniques, using the occluded versions of six different facial datasets. The proposed method achieves accuracies between 69.24% and 99.46% depending on the dataset, and always higher than the compared methods. For the face classification task, we compared the performance of OSF-DNS with the features obtained by four deep learning techniques. Experimental results on LFW and CFPW datasets showed that the proposed hashing method outperformed the results obtained with deep features with an accuracy up to 89.53%.Highlights •Victims in Child Sexual Exploitation Material (CSEM) often present the eye region occluded.•Deep learning face recognition models fail when the eye region is occluded.•We present a perceptual hashing method called OSF-DNS.•OSF-DNS is robust against eye occlusion for face verification and classification.•OSF-DNS outperforms other perceptual hashing and state-of-the-art face recognition methods.