Multi-stream GCN for Sign Language Recognition Based on Asymmetric Convolution Channel Attention

Sign Language Recognition (SLR) has become increasingly important all over the world. In this paper, we propose the Asymmetric Convolution Channel Attention module (ACCA), which uses multi-scale vertical and horizontal skeletons of convolution kernel to extract the channel characteristics. We argue that the ACCA avoids some redundant features and extracts more distinctive features. It also enhances the robustness of vertical and horizontal flipping. The backbone of our network is the SL-GCN and the dataset is the WLASL2000. As a result, the top1 accuracy of joint, bone, joint-motion, and bone-motion streams are 48.12%, 47.16%, 31.10%, and 30.09% respectively. After multi-stream fusion, we achieve 53.52%. We also put the ACCA into other models and prove the generalization. Furthermore, we achieve 54.12% based on MS-G3D, which exceeds the most advanced accuracy of the WLASL2000 dataset using the single-modal skeleton-based model.