Pattern unlocking guided multi-modal continuous authentication for smartphone with multi-branch context-aware representation learning and auto encoder

Widely accepted explicit authentication protocols are vulnerable to a series of attacks, for example, shoulder surfing and smudge attacks, leaving users with the constant burden of periodic password changes. As such, we propose a novel framework for continuous authentication on smartphones. This approach is guided by pattern unlocking, which is widely used and will not cause learning cost. After collecting multi-modal data that describe both behavioral and contextual information, we employ a multi-branch context-aware attention network as the representation learner to perform feature extraction, then an auto encoder is then used for authentication. To overcome challenges, including cold-start and few-shot training, which is less discussed in other works, we incorporate transfer learning with a coarse-to-fine pre-training workflow. Additionally, we deploy a hierarchical approach to offload model tuning overhead from smartphones. Extensive experiments on more than 68 000 real-world recordings validate the effectiveness of the proposed method, with an EER (equal error rate) of 2.472% under mixed contexts, which consistently outperforms state-of-the-art approaches under both static and mixed contexts. Integration of context-aware representation learner and self-supervised auto encoder revolves continuous authentication performance. Transfer learning driven coarse-to-fine training addresses cold-start/few-shot problem and accelerates actual application. Test bed with more than 68k real-word samples shows our work achieves 2.472% EER under mixed contexts, outperforming state-of-the-art.image