1 Introduction

In 2020, COVID-19 has been reported as a worldwide pandemic by the World Health Organization (WHO). The pandemic has affected more than 200 countries, with approximately two million death [23, 33]. Also, COVID-19 has resulted in 97,966 of businesses to permanently shut down during the pandemic and destroying millions of job opportunities around the world. Recently, two COVID-19 vaccines have been received an approval from WHO: BioNtec/Pfizer and Moderna [21]. According to the Food and Drug Administration, those COVID-19 vaccines provide strong protection against COVID-19 within about 10 days of the first dose [6, 11, 28].

Fig. 1
figure 1

A Samples showing sentiment translation for each tweet from the collected dataset

Full size image

Social networks such as Twitter, Facebook and Instagram have become an important part of our daily life. it provides interfaces for individuals to use their own terms to discuss emerging topics and express their emotions and feeling towards issues happen all around the world. Twitter,for example, is a free broadcast channel that offers registered users to talk and communicate with others using 140 character texts. According to Twitter latest report [10], A total of 1.3 billion accounts have been created and there are roughly 330 million monthly active users and 145 million daily users who post more than 500 million tweets every day [27].

Recent studies show that there is a growing tendency for people to talk about COVID-19 pandemic in these online channels. With the recent released of Pfizer vaccine, people start posting a lot of rumors regarding the safety concerns of the vaccine especially among the elderly people [9, 32, 37]. In fact, people feel uncomfortable and fair due to vaccine uncertainties and the death of 23 people who received the vaccine [35]. Figure 1 presents an example taken from Twitter that shows a number of users who are depressed and fair toward the Pfizer vaccine.

Emotion analysis of users tweets aims to the use of machine learning approaches to analysis the text and the people attitude and feelings toward different topics. Many traditional machine learning approaches has been proposed to detect people opinions on social contents [2]. However, these approaches heavily relies on incomplete handcrafted features, which in result produce an accurate outcomes.

Deep Learning have recently been applied wide range of domains such as computer vision and pattern recognition [1, 7, 18]. Comparing with traditional machine learning approaches, deep learning have already made impressive outcomes in in many fields such as Natural Language Processing (NLP), machine learning and emoticon analysis. Many studies show that deep neural networks outperformed more traditional machine learning in term of accuracy and the detection of people feelings [14, 22].

The primary objective of our study is to provide analysis of the users emotions through the tweets regarding to COVID-19 vaccine. The research begins with the collection of tweets regarding COVID-19 vaccine followed by extensive cleaning of the dataset. The dataset contains around 50,000 tweets that have been published from the release of the vaccine till 1st January 2021. During training the model, the tweets are tokenized into words and then transformed into a feature vector. The main model developed and tested for the framework are Long Short-Term Memory (LSTM), Naive Bayes, decision tree and SVM. Natural language Toolkit (NLTK) library has been utilized which goes about as a suitable content processor for language dealings. Finally, WordCloud is presented to show frequently used words that describe users emotions. The approach is termed as “unigram”, for two words it is “bigram”, for three words it is “trigram”, respectively. To evaluate our model, we compare it with other models as Support Vector Machine (SVM) and Naive Base(NB), the results show that our model is more accurate and gives an efficient results.

The main contributions of our research work include:

  • To the best of our knowledge, this is the first paper that analysis and identify the people opinion toward the COVID-19 vaccine.

  • We have introduced a deep-leaning model that analysis and understand people feelings, and capture the frequent used words on social content.

  • We have executed different state-of-the-art deep learning classifiers to extricate individuals feelings in social media during the new period.

  • The main significance of this research work is that since we are using more than one language in predicting users opinion and emotion understanding in social media regarding COVID-19vaccine, it gives more flexibility and accurate results when compared to its single language counter part.

This paper is organized as follows: Section 2 presents the related works. Section 4.1 presents dataset description. Section 3 introduces the proposed methodology. Section 4 presents the experimental observations and detailed discussion. Finally, the conclusions and possible future works are introduced in Section 5.

2 Related work

As Novel Corona Pandemic COVID-19 reports a major warning to the international population health, collaboration is needed by all the nations to combat it [24]. The media and other web-based media channels should morally introduce the pertinent and right reports to expand inspiration among the overall population as opposed to introducing one-sided data; such coverage may only serve to further divide people and cause fear [16, 31].

Emotion analysis is the social occasion of individuals’ perspectives with respect to any any event happening in real life [12, 26]. In such circumstances in which the world is at present experiencing, understanding the feelings of the individuals stands critical. The grave situation wherein individuals can’t leave their homes requests investigating what the individuals is really being thinking about the entire situation. Thus, the authors have arranged this work on understanding the requesting circumstance particularly on social media [5, 29].

Merchant [25] explained the significance of spreading data through different social media stages during the outbreak of COVID-19. It discussed about that moving right and dependable data is fairly significant for the individuals if there should arise an occurrence of emergencies as online media is the indicative device and the most extreme reference framework. Individuals accept more on presents on when on get tried, how to manage the outcomes, and where to get care. Along these lines, exploring incorrectly and misdirecting data can in some cases be risky to somebody’s life. It talked about the significance of correspondence as it is straightforwardly relative to individuals’ conviction and response as needs be.

Li [20] explored the effects of COVID-19 on the psychological wellness of individuals by directing different tests on opinion examination utilizing long range informal communication locales. It was shown by tests that the information holes of diminutive term people change in mental conditions after the outbreak.

Sharma [30] conducted an inside and out investigation on shifted subjects comprising of assumptions and patterns with distinguishing bogus data and designated spots in spreading bogus data on pandemic COVID-19 through Twitter. The paper leaves open stamping remark for misrepresentation to improve classification techniques [19].

Previous researchers were focused on developing methods to achieve accurate for prediction of the spread of COVID-19 through twits. The main drawbacks of the previous research works were, they did not mention anything study to provide analysis of the emotions through the tweets regarding to COVID-19 vaccine. Therefore, this paper analysis and identifies the people opinion toward the COVID-19 vaccine and helps decision maker to improve and indicate an ascending trend for the cases in the future.

3 Methodology

Understand the user emotions in social media has gain a lot of attention from researcher recently. The goal of this study is to the impact of social media on the emotions and feelings of people regarding to COVID-19 vaccine. Emotion analysis can be used here to identify expressed emotions from unstructured texts To efficiency understand the social media impacts, we propose a deep learning based model to understand people emotions. Figure 2 illustrates the framework architecture of our model. The proposed model consists of three main components namely: (1) Emotion Labelling; (2) Pre-processing; and (3) Emotion Classification. In the following sections, we will explain those components in details. The detailed steps for the proposed model are shown in Fig. 2.

Fig. 2
figure 2

The system architecture of the proposed model

Full size image

3.1 Emotion labeling

The problem we address in this paper is to understand the people feeling toward the covid-19 vaccine. As shown in figure, the input to this component is a set of tweets regarding to COVID-19 vaccine crawled from Twitter API using the following hashtags: COVID-19 vaccine, vaccine, and Pfize. Then we asked PHD students to label the data in with 1 or 0 as positive or negative.

3.2 Pre-processing

Pre-processing step play an important role in our mode. To improve the quality of our model, the pre-processing step is done to clean the tweets. In this step, we first combine all tweets in Comma Separated Values (CSV) file has the tweets and it is cleaned by removing duplicate rows or similar tweets. Also, the tweets are cleaned by deleting the associated and redundant symbols such as @, RT, #, URLs, numeric values and punctuation marks. Initially,tokenization for words is done and counting the top frequent words, then the relevant features are extracted from the text available in the tweets to build and train the classification model.

3.3 Emotion classification

The problem we address in the paper is to understand the people emotions expressed regarding COVID-19 vaccine. This step receives the labelling data as input and then the feature extraction process is performed to capture the most important words.

In text mining, TF-IDF, stands for term frequency–inverse document frequency, is a statistic measure that evaluates how relevant a word is to a single document in a corpus of documents. The core idea behind TF-IDF is that each word that appear in a document has two scores: term frequency (TF) score and inverse document frequency (IDF) score. TF computes the number of times a term appears in a document, while IDF is the inverse document frequency of the word across a collection of documents. Multiplying those two score gives the TF-IDF weight of that term [34], where TF-IDF here reflects the weight of a term in a particular document. The higher the score of TF-IDF weight, the more important that term is in that document. In the proposed model, we apply the TF-IDF measure to extract and model different words in our dataset. The formula of TF-IDF is shown by the following equations [3, 34]:

$$\begin{aligned} TF(t)= & {} \frac{\text {Number of times term t appears in a document}}{\text {Total number of terms in the document}}. \end{aligned}$$
(1)
$$\begin{aligned} IDF(t)= & {} \log _e\frac{\text {Total number of documents }}{\text {Number of documents with term t in it}}. \end{aligned}$$
(2)

Subsequently, the TF-IDF value can be calculated by taking a product of the two statistics: TF and IDF. Intuitively, this means:

$$\begin{aligned} TF-IDF(t) = TF(t) * IDF(t) \end{aligned}$$
(3)

As the machine learning or deep learning models do not understand the text data. These algorithms usually deal with numbers, and natural language is text data. So text data need to transform into numbers, this process known as text vectorization. It is a fundamental step in the process of machine learning for analyzing data, and different vectorization algorithms will drastically affect final results. A term will be considered highly relevant in a document and assigned a higher weight when it has a high TF value (in that document) and a low document frequency in the whole corpus. The IDF value in that situation will be greater than 1 (since the log value is greater than 1), and thus the overall TF-IDF value will be large. After words transformed into numbers, the TF-IDF score can be fed to machine learning algorithms such as Naive Bayes and Support Vector Machines.

TF-IDF is useful in our task as it provides a vector representation of tweets. Once we computed the TF-idf for our dataset, the next step aims at analysing the users emotions toward the vaccine. In this work, we have used a deep learning model namely LSTM which is a variant of a Recurrent Neural Network (RNN) and it is a powerful tool in predicting sequence and time-series data-related problems. LSTM model was selected in this work as COVID-19 dataset is also time-bound. The LSTM has a special feature that enables to store previous information and predict the future trend of the virus under consideration. The LSTM model takes in the input features and predicts the class label. The LSTM model that we have used in this study has the following configurations. It uses 300 epochs and a loss function namely sparse categorical crossentropy with adam optimizer. The softmax activation function is used in output layer. The total number of tunable parameters used in the proposed model are summarized in Table 1.

Table 1 The description of the proposed LSTM model
Full size table

Finally, the softmax activation function is used to classify the people emotions to either positive or negative. The mathematical form of this activation function is presented in Eq. 4.

$$\begin{aligned} S(x_i) = \frac{\exp ^{x_i}}{\sum _{j=1}^{n}\exp ^{x_j}} \end{aligned}$$
(4)

where \(x_i\) is input vector, n is number of classes in the multi-class classifier and \(\exp ^{x_i}\) is standard exponential function for output vector. The output of our model is discussed in details in the following section.

4 Experimental results and evaluation

The model aims to classify emotion into positive or negative scores. During training the model, the tweets are tokenized into words and then transformed into a feature vector. The main models developed and tested for the framework are Long Short-Term Memory (LSTM), Naive Bayes, decision tree and SVM. Natural language Tooklit (NLTK) has been widely used recently as it provides an implantation of various traditional and advanced machine leaning algorithm. Hence, in our experiment, we use the available machine leaning mentioned above in NLTK to compare the result of our model.

In the following subsections, we first discuss the dataset used in this paper and then demonstrate the experimental results in term of simulated results and tabular form. We mainly compare the output of model with well-known approaches and the results show that our model can successfully capture the people emotions expressed in social contents

4.1 Dataset description

This section describes the collected dataset from Twitter. The dataset contains around 50,000 tweets that have been published from the release of the vaccine till \(1^{st}\) January 2021. Further, The pre-processing step is done to make text lowercase,remove punctuation emoji’s and stop-words. After that, the labeling step is applied to annotate the tweet as positive or negative.

The most frequent words used in the dataset to show people feelings is visualized using the WordCloud figures [4]. These figures display words that do not prove any efficiency in representing a viable solution during pandemic. Figure 3 shows the wordcould for the top words of Twitter data regards hope and fair toward the Pfizer vaccine. These top frequent words can indicate the common words used in social media to describe people feelings towards COVID-19 vaccine.

Figures 4 and 5 show the wordcloud top words of Twitter datat regard hope and fair toward Pifzer vaccine. The top frequent positive words for example are: (safe), (trust), and (victory). While, the top negative words are: (danger), (infected) and (fainting).

Table 2 shows the top frequent words in the whole dataset. The top frequent words are (Vaccine) and (Corona). While the word ( Pfizer) is repeated 6380 times in the dataset. Figure 6 presents the distribution of the positive and negative labels. From this figure, it is clear that the majority of labels is positive regards the COVID-19’s vaccine.

Fig. 3
figure 3

The wordcloud for the top words of Twitter data

Full size image
Fig. 4
figure 4

The wordcloud for top negative Emotion

Full size image
Fig. 5
figure 5

The wordcloud for top positive words

Full size image
Table 2 The top 8 words in the dataset
Full size table
Fig. 6
figure 6

The distribution of the positive(1) and negative(0) labels

Full size image

4.2 System evaluation

The classification accuracy is not the best metric in the case of class imbalanced datasets, and thus we chose to use additional performance metrics to mitigate the issue. Therefore, four additional performance metrics are used as additional performance measurements such as precision, sensitivity (recall), F1-score, and ROC curve. The Eqs. 5, 6, 7, and 8 are the mathematical expressions of Accuracy, F1-score, recall, and accuracy determined on the basis of True Positive (TP), False Negative (FN), False Positive (FP), and True Negative (TN) test data samples. [13].

$$\begin{aligned} Accuracy= & {} \frac{TP + TN}{TP + FP + FN + TN} \end{aligned}$$
(5)
$$\begin{aligned} F1-score= & {} 2 \frac{precision\, \star \,recall}{precision+recall} \end{aligned}$$
(6)
$$\begin{aligned} Recall= & {} Specificity = \frac{TP}{TP + FP} \end{aligned}$$
(7)
$$\begin{aligned} Precision= & {} \frac{TP}{TP + FN} \end{aligned}$$
(8)

4.3 Machine learning results

Experiments are carried out using a dataset obtained through the following hashtags vaccine, corona, coronavirus, and COIVD-19. The collected dataset includes around 50,000 tweets. To fit the model, the final version of the transformed data split into train and test sets such that 70% data for the training set, 30% for the testing set. To evaluate the performance of the selected classifiers, unigram, bigram, trigram and word embedding using word2vec have been performed.

After the texts have been changed into vectors, classification are performed. This study includes hyper-parametric classifiers from k-nearest neighbour (KNN) [17], Naïve Bayes (NB), AdaBoost (ADB), Decision tree (CART) [8], Random Forest (RF) [13], Support Vector Machine (SVM) [36], and Linear Regression Classifier (LR) [15].

In general, Linear classifiers such as SVC, logistic regression, and LR are more efficient for text data classification tasks. Also, the linear method includes linear functions to forecast the input features. Therefore, to achieve high accuracy, a strong classifier must be developed. The K-fold Cross-Validation technique is used to calculate the average accuracy metric of the model by creating many different samples of the data. Moreover, The k-fold is a common form of cross-validation technique in which, for example, if \(k=10\), 9 folds are used for training and 1 fold is used for testing the model, and this process is repeated until all folds get a chance to be the test set one by one as shown in Fig. 7. The performance measure reported by k-fold cross-validation is then the average of the values computed in the loop. Thus, it gives us a good idea and clear understanding of the model’s generalization ability, which is particularly useful when we have small data and can not afford to break it into training and testing sets.

Fig. 7
figure 7

The K-fold Cross-Validation technique

Full size image

Table 3 shows the values of recall, precision, and F1-score for the SVM model. The proposed SVM model used for the determination of the Emotion for class 1 outperforms class 0. Moreover, the proposed method is biased due to it exhibits high sensitivity towards major class 1. Also, to calculate the comprehensive measurement of the performance, F1-score is used to compare the accuracy of the technique. From the performance measures values, it is clear that the SVM approach provided remarkable results in predicting Emotion from tweets.

Table 3 The performance of SVM model
Full size table

Similarly, Table 4 presents the values of Precision, Recall, and F1-score for the experimental results of the Naive Bayes model. The Naive Bayes model used for classifying the Emotion into two classes namely class 1 and class 0. The Naive Bayes method is also biased towards major class 1. Finally, in Table 4 the performance of the Naive Bayes model gives maximum accuracy of 79%, the precision of 78% and F1-score of 79%.

Table 4 The performance of NB model
Full size table

Figure 8 presents the comparison of machine learning methods. The comparative study is carried in the following Machine learning models namely Logistic Regression (LR), K-Nearest Neighbors (KNN), AdaBoost (ADB), Decision Tree (CART), GaussianNB (NB), Random Forest (RF), and SVC (SVM). The performance of each model is assessed utilizing arrangement measurements such as precision, recall, f1-score and accuracy on the test dataset. Based on the experimental results of the LR model, it is outperforms compared with the other models.

Fig. 8
figure 8

Comparison of machine learning methods

Full size image

The outcomes in Fig. 8 show an improvement in the accuracy when a comparison is made between the different existing models. Improvement in the results of performance comparison can be enhanced through utilizing grid search strategies to select the most effective of the proposed techniques.

4.4 Long short term memory (LSTM)

Table 5 presents the accuracy achieved by the state-of-the-art LSTM model being actualized on the gathered information. For example, all tweets are represents using tfidf method and implemented with the unigram. Because of the dataset has different numbers of positive, and negative tweets, LSTM model gives the highest accuracy of 98% for training set and 73% testing set with unigram under the Tfidf technique. Figures 9, 10 show the plot for the loss and accuracy of the LSTM model.

Table 5 Long Short Term Memory (LSTM)
Full size table
Fig. 9
figure 9

The loss values of LSTM model for the training and testing sets

Full size image
Fig. 10
figure 10

The accuracy of LSTM model for the training and testing sets

Full size image

5 Conclusion

This paper has provided an analysis of the emotions through the tweets regarding to COVID-19 vaccine. The release of COVID-19 vaccine brings out the fact that a lot of people in social media starts talking about safety of vaccine. People often use specific words to express their feelings and emotions towards any topics they discussed in social media. Hence, in this paper, we present a LSTM model that provides an analysis of such emotions expressed by people regarding the COVID-19 vaccine. The proposed model first received the Twitter data as inputs. Then, the classifying process is performed on the tweets to identify the people emotions. A LSTM model then is used to train the data and classify the people opinion about COVID-19 vaccine to either positive or negative. We evaluate our model in Twitter dataset, and the result shows that our model outperforms the current approaches for emotion analysis in term of accuracy. Our proposed deep learning classifiers has achieved accuracy up to 98% for training set and the accuracy for testing set is 73%. In the future work, we will focus on applying the proposed model in other type of vaccine.