BERT-Based Models with BiLSTM for Self-chronic Stress Detection in Tweets

Abstract

Recently, the swift increase of social media platforms has provided a rich source for studying several users’ psychological phenomena. For instance, stress identification in text content can lead to knowing some insights about social media users’ mental health. Actually, chronic stress has a huge negative impact that requires the development of different methods for early detection and diagnosis. In this paper, we propose a deep learning approach and a Natural Language Processing (NLP) method to reveal self-reported chronic stress from tweets. In effect, we have implemented distinct pre-trained BERT (Bi-directional Encoder Representations from Transformers) embedding models, along with a BiLSTM (Bidirectional Long Short-Term Memory) classifier. Actually, we fine-tuned the pre-trained BERT models by leveraging their powerful contextual representation. Next, the output of the embedding is fed into a BiLSTM model which further refines the stress classification by capturing the sequential dependencies in the tweet text. Experiments disclosed that BERT with Talking-Heads Attention architecture is the best model for such text classification tasks. Moreover, our suggested model has achieved good performance and surpassed the baseline architectures for chronic stress detection in Twitter data.