Hidden Markov Models for feature-level fusion of biometrics on mobile devices

Although biometrics have forayed into the mobile world, most current approaches rely on a single biometric modality. This limits their recognition accuracy in uncontrolled conditions. For example, performance of face and voice recognition systems may suffer in poorly lit and noisy settings, respectively. Integration of identifying information from multiple biometric modalities can help solve this problem; high-quality identifying information in one modality can compensate for the absence of such information in a modality affected by uncontrolled conditions. In this paper, we present a novel multimodal biometric scheme that uses Hidden Markov Models to consolidate data from face and voice biometrics at the feature level. An implementation on the Samsung Galaxy S5 (SG5) phone using a dataset of face and voice samples captured using SG5 in real-world operating conditions, yielded 4.18% and 9.71% higher recognition accuracy than face and voice single-modality systems, respectively.