Deep Learning For Language And Vision Tasks In Surveillance Applications

The keyword spotting and handgun detection tasks have been widely used to manipulate devices and monitor surveillance systems in a more efficient manner. In spite of the advances of deep learning approaches dominating those tasks, the effectiveness of them is mostly tested and evaluated in datasets of exceptional qualities. This paper aims to analyze the performance of these tools when information captured by common devices is used, for example; commercial surveillance systems based on standard resolution cameras or microphones from smartphones. For this, we propose to build an audio dataset consisting of speech commands recorded from mobile devices and different users. In the audio section, we evaluate and compare some state of art keyword spotting techniques with our own model, which outperforms the baselines and reference approaches. In this evaluation we obtained an accuracy of 83%. For the handgun detection, we did a fine tuning of YOLOv5 to adapt the model and perform the detection of handguns in images and videos. This model was tested on a new dataset that uses labeled images from commercial security cameras.