A CONCEPTUAL ENHANCEMENT OF LSTM USING KNOWLEDGE DISTILLATION FOR HATE SPEECH DETECTION

Abstract

Hate speech is an issue to most governments and the public's concern due to the increased emergence of social media platforms and the increasing use of such media to disseminate hate speech to individuals, groups of persons, communities, or races. Hate speech is also by no means always on the rise due to the high rate of remote service usage such as communication, online studies, meeting, dating, etc. With the recent outbreak of COVID-19, there has been an increase in the number of users on different social media platforms. This increase in number has brought about an increase in issues such as hate speech, among others. Therefore without detection and analysis of hate speech, one cannot imagine social media to be free of malicious content. Deep Neural networks inspired by the human brain's work have continuously demonstrated their importance and relevancy in many different I.T. fields, particularly hate speech detection. This research aims to provide a detailed process of improving LSTM used for hate speech detection using knowledge distillation. The knowledge transfer is done from the more extensive network (teacher) to the smaller student network. The teacher has trained for five full epochs to output accuracy of 76.8%, the student network trained from the teacher network for three entire epochs attained an accuracy of 82.6%. Another student model cloned and trained from scratch for three full epochs instead of the teacher network achieves an accuracy of 75.4%.