Improving Keyword Search Performance in Sign Language with Hand Shape Features

Handshapes and human pose estimation are among the most used pretrained features in sign language recognition. In this study, we develop a handshape based keyword search (KWS) system for sign language and compare different pose based and handshape based encoders for the task of large vocabulary sign retrieval. We improved KWS performance in sign language by 3.5% mAP score for gloss search and 1.6% for cross-lingual KWS by combining pose and handshape based KWS models in a late fusion approach.