SENTIMENT CLASSIFICATION ON SUICIDE NOTES USING BERT, BI-LSTM, AND MULTI-HEAD ATTENTION

Abstract

Nowadays, there is a lot of focus on mental health and mental illnesses. Suicide is a topic of serious concern and a lot of effort is being put in by researchers and mental health professionals to combat this issue. “Sentiment Analysis on Suicide Notes” is a lesser-explored area. Understanding the emotions of the people committing the act of suicide is crucial to rage a fight against suicide and to develop areas of suicide prevention. This study focuses on works that relate to the task of “Sentiment Analysis on Suicide Notes”. Despite the importance of this task, there aren’t enough datasets available to train models to identify different emotions in suicide notes. A variety of emotions are being represented in the dataset I am using. They are ”happy”, ”sad”, ”neutral”, ”love”, ”hate”, and ”proud”. In this study, I will train the model in three phases. First, I will extract features from the input sentences using Bidirectional Encoder Representations from Transformers (BERT). This will be followed by sequence modeling performed using the Bidirectional Long Short-Term Memory (Bi-LSTM) network. In the third phase, to emphasize more attention on the important segments of the sentence, I’ll use a Multi-head Attention mechanism. This would help increase the overall classification efficiency as it combines the strength of three different components. Resulting with Precision, Recall, and F1-Score all at 81%, this approach proves to be an effective one. This will help study the emotional content of the message written in suicide notes and will surely help in promoting initiative toward mental health in both personal and professional contexts.