Hate speech detection on Twitter using transfer learning

Social Media has become an ultimate driver of social change in the global society. Implications of the events, that take place in one corner of the word, reverberate across the globe in various geographies. This is so because the huge amount of data generated on these platforms, reaches the far corners of the world in the blink of an eye. Developers of these platforms are facing numerous challenges to keep cyber space as inclusive and healthy as possible. However, in recent years, the phenomena of offensive speech and hate speech have risen their ugly heads. Despite manual efforts, the scope of this problem is so immense that it cannot be tackled by using concerted teams. In fact, there is a need that an automated technique is designed that detects and removes offensive and hateful comments before the materialization of their harmful impacts. In this research work, we develop an Urdu language hate lexicon, on the basis of this lexicon we formulate annotated dataset of 10,526 Urdu tweets. Furthermore, as baseline experiments, we use various machine learning techniques for hate speech detection. In addition, we use transfer learning to exploit pre-trained FastText Urdu word embeddings and multi-lingual BERT embed dings for our task. Finally, we experiment with four different variants of BERT to exploit transfer learning, and we show that BERT, xlm-roberta and distil-Bert are able to achieve encouraging F1 scores of 0.68, 0.68 and 0.69 respectively, on our multi class classification task. All these models exhibited success to varying degree but outperform a number of deep learning and machine learning baseline models.