Detecting and recognizing faces from video images using multi-deep CNN based rank-level fusion

The advent of deep neural networks has provided a competitive edge in facial analysis efforts, especially the detection and recognition of faces from video images. This paper outlines a methodology wherein a face database is created from an outdoor video clip, and a rank-level fusion mechanism has been proposed for face recognition using a database that makes use of the probabilities produced by the softmax functions of multiple deep convolutional neural network (CNN) models. These probabilities are referred to as certainty indices for the potential classes to which a test sample may belong. For each of the CNN models, fuzzy ranks are obtained for a subset of classes with the top certainty indices by applying a Gaussian distribution function followed by a complementation operation. For each class, appearing in one or more subsets corresponding to deep network models, average of the certainty indices is complemented and fused with the sum of complemented fuzzy ranks. While calculating the cumulative sum of the fuzzy ranks, a penalty, is foisted to the class for not being present in any subset. This can avoid the chance of becoming an unexpected winner. The class with the lowest final ranking was identified as the class pertaining to the test sample. In addition to testing with the custom-made outdoor video-based face dataset, the projected method was evaluated with a standard surveillance indoor dataset, ChokePoint. Considering all test sets on ChokePoint dataset, the average recognition accuracy of the proposed method is 99.54%. The same for custom-made dataset is 93.96%. For both datasets, precision and macro-average scores of the proposed method in all cases are higher than the individual deep CNN models.