The data used to perform the experimental evaluation comes from a public repository that contains a real-time collection of tweets related to the 2020 US presidential election from December 2019 to June 2021 (Chen et al. 2021). From such repository we considered only the tweets published close to the election event (from September 1 to October 31, 2020), i.e., about 160 million of which 18 million are tweets (11%), 110 million are retweets (69%), and 32 million are replies (20%), posted by about 29 million users. Only 22% of filtered data contain hashtags (e.g., \(\#trump2020\), #bidenharris2020), useful to understand the arguments used in favor of the different candidates. In particular, the percentage of tweets published with at least one hashtag related to Trump (i.e., \(\#trump\), \(\#trump2020\), and \(\#maga\)) and Biden (i.e., \(\#bidenharris2020\), \(\#biden\)) is about 31% and 11%, respectively. However, 7% of tweets contain at least one negative hashtag about Trump (i.e., \(\#trumpknew\), \(\#pedotrump\), \(\#trumphascovid\), \(\#trumptaxreturns\), #bountygate), whereas only 1% of tweets contain a negative hashtag for Biden (i.e., \(\#crookedjoebiden\)). In order to ensure the representativeness of the collected posts, we analyzed users’ account information, filtering out content posted by users that show an anomalous publishing activity or inconsistent profile information. This step allows to avoid the negative effects caused by the presence of content published by new sites and social bots, which can introduce a heavy bias in social media data (Cantini et al. 2022). We further analyzed the publishing behavior of the users in the filtered dataset by determining the Complementary Cumulative Density Function (CCDF) of shared tweets per user. Specifically, given the random variable X representing the number of shared tweets, it is determined by the frequency of users publishing a number of posts greater than x, i.e., the probability \(P(X > x)\). The scatter plot in log-scale shown in Fig. 7, reveals a highly skewed distribution, with few active Twitter users posting a huge amount of tweets, and many users posting infrequently or not at all, the so-called social lurkers.
Statistical significance of collected data
Here we investigate the statistical significance of the collected data in order to assess users representativeness, i.e., whether they can be considered voters of the political event under analysis.
Firstly, from tweets metadata we extracted aggregate information on the used language of social media users, discovering that most of tweets have the lang field set to English (about 90%), whereas the remaining 10% is Undefined or set to other languages like Spanish. Secondly, we compared the number of Twitter users in our dataset, grouped by state, with the number of adult citizens actually living in that state, belonging to the voting-eligible population (VEP).Footnote 6 Specifically, users were associated with states via Twitter metadata, by analyzing the location field present in each tweet, which indicates the location defined by the user in his/her Twitter account (e.g., Austin, TX). It is worth noting that, from the textual analysis of this field, it is not always easy to extract a meaningful city/state, as many users either left the field blank, or did not provide precise information (e.g., “USA”), or specified fictitious or nonexistent locations (e.g., “the moon” or “NY, Italy”). We measured the strength of this correlation, finding a Pearson coefficient \(r=0.97\), significant at \(p < 0.01\). The linear relationship that links users and the voting-eligible population can be easily seen in Fig. 8, which depicts an interpolation of the related scatter plot, with a goodness-of-fit \(R^2=0.93\). Notice that outlier states were not considered in this step in order to achieve meaningful results, by excluding data of different magnitude. In addition, we explored age and gender distribution of analyzed users, finding out that about 94% of them are adults (at least 18 years old)Footnote 7 and almost equally divided by gender .Footnote 8
Among all the available tweets we have selected those published by users located in the 11 main swing states (i.e., Arizona, Florida, Georgia, Michigan, Minnesota, Nevada, New Hampshire, North Carolina, Pennsylvania, Texas, Wisconsin). We analyzed only these states as they are characterized by a marked political uncertainty and their outcomes have a high probability of being a decisive factor of the electoral event. We made this data, used in all the subsequent analysis steps, publicly available on Github.Footnote 9
Table 1 reports a comparison between the users we were able to capture for each swing state and the VEP. The high correlation between the number of analyzed users per state and the VEP leads to a significant set of social media data effectively exploitable to determine the polarization of public opinion. However, despite the representativeness of the considered posts, the results achieved by the analysis of the online conversation can be influenced by platform biases. Specifically there exist usage biases due to the distribution of users of a social media platform in terms of gender, age, culture and social status, as well as technical biases related to platform policies about data availability and restrictions imposed in some areas of the world.
Trending topics of the election campaign
In this step we identified the main politically related discussion topics characterizing the 2020 US election campaign. Achieved results are shown in Fig. 9, where six clusters are clearly visible, each one related to a different topic of discussion. Moreover, Table 2 summarizes the discovered topics by reporting the corresponding top hashtags.
The first topic is focused on the criticisms leveled at Trump regarding the management of the health emergency in the USA caused by Covid-19 pandemic. The second one is related to the online discussion about town hall meetings, covering different topics against Trump like discrimination, veterans and climate crisis (e.g., he referred to climate change as a “hoax”, and to veterans as “human scum”). The third one is a general topic about the presidential election. The fourth topic is related to the accusations of corruption and wrongdoing in regards to China and Ukraine leveled against Hunter Biden, i.e., the son of the democratic candidate Joe Biden. The fifth topic focuses on the nomination of the conservative Amy Coney Barrett for a seat on the Supreme Court as successor to the liberal Associate Justice Ruth Bader Ginsburg. Finally, the last topic is related to the online discussion of Trump’s supporters, characterized by notorious hashtags like #maga or #kag.
Once the major discussion topics were detected, we analyzed their overall impact on the online conversation, along with their evolution in the eight weeks included in our observation period, as shown in Fig. 10. In particular, we calculated the volume of each hashtag-based topic by determining the percentage of tweets that contain hashtags belonging to the corresponding cluster. Considering our overall observation period, the most relevant topic is about Covid-19 pandemic and it specifically refers to Trump’s mismanagement of the health emergency. Other topics are related to the presidential election in general or arise from the publishing activity of Trump’s supporters. Also Biden’s supporters significantly contributed to the online discussion, by leveraging anti-Trump sub-topics that have emerged from several town hall meetings, about discrimination, veterans and the position of the Republican candidate about the climate crisis.
For what concerns the temporal evolution of the detected topics, we found that in the early weeks online conversation focused on the relationship between Trump and Covid-19 pandemic. In addition, the discussion about the US Supreme Court showed a slight increase close to the nomination, announced by Donald Trump, of Judge Amy Coney Barrett as Associate Justice of the US Supreme Court to fill the vacancy left by the death of Ruth Bader Ginsburg. In the following weeks, the focus of the online conversation shifted to various topics related to the approach of the Election Day and the importance of voting. We also observed an increase in the volume of tweets concerning the accusations leveled against Joe Biden’s son (i.e., Hunter Biden), a topic discussed mostly by the Democratic candidate’s detractors. Finally, other topics regarding the support voters expressed toward Trump and their criticisms leveled against him linked to town hall meetings showed an almost constant impact on the online conversation.
In this step we investigated the temporal dynamics of social media conversation, in order to analyze users’ publishing behavior, studying how it is related to the detected polarity and how it reflected the occurrence of external events (e.g., debates, rallies, etc.). However, as described by the repository owners in Chen et al. (2021), there may be gaps in the dataset due to several issues. Firstly, the data collection step was highly contingent upon the stability of the network and hardware. Secondly, Twitter significantly limits the number of tweets that can be rehydrated. Finally, tweets may no longer be available as users have been removed, banned, or suspended.
Figure 11 shows the timeline of polarized tweets volume annotated with the four main political debates occurring during the election campaign, i.e., between September 1 and October 31. The first observation period (September 1 to September 28) exhibits significantly different communication dynamics prior to the first debate. Interestingly, this image shows an intense activity spikes of Biden’s supporters, as a likely consequence of President Trump’s actions:
September 10: president Trump has attacked Democratic Vice Presidential candidate Kamala Harris.
September 15: despite being banned by state authorities from holding rallies, President Trump still decided to hold one in Nevada.
September 18: president Trump blamed blue states for the high number of the US Covid-19 fatalities.
September 28: during a rally in Pennsylvania, Trump called Biden “a dishonest politician and a puppet in the hands of the radical left”.
The second and third observation windows (from September 30 to October 31) show typical weekly cycles of social media chatter, with no particular explosion or shock-related spike from external events, except for October 6 (before the Vice Presidential debate) and October 13 (before the second Presidential debate).
Comparative analysis with opinion polls
In this step we assessed the effectiveness of our approach in determining the polarity of social media users with the aim of understanding which candidate or party public opinion is most in favor of. A first remarkable result was obtained through a real-time analysis, carried out on Twitter data collected during the two weeks before the Election Day. Specifically, IOM-NN was able to correctly determine Joe Biden’s lead over Donald Trump, especially in Georgia, where a Democratic candidate had not won since 1992 with the election of Bill Clinton. This promising result, publicly available through our university web portal,Footnote 10 represents a step forward with respect to our previous work, as it gives a clear proof of the real-time effectiveness of IOM-NN, which suggests the possibility of using it to enhance or even replace traditional opinion polls.
Starting from the encouraging real-time results, we extended that analysis by focusing on the main eleven swing states, as described in Sect. 4.1.1. Specifically, we compared the results obtained through IOM-NN with the average values of the latest opinion polls before the election.Footnote 11 For each analyzed state, Table 3 reports the real voting percentages, opinion polls, and IOM-NN estimates. The two candidates (i.e., Joe Biden and Donald Trump) are indicated with “B” and “T”, respectively. The winning candidate is written in bold when it is correctly identified.
The results of the comparison are summarized in Fig. 12, which shows that the estimates achieved by IOM-NN, related to the voting intentions of social media users are more in-line with the actual behaviors of voters with respect to the opinion polls, thus giving a clue to the final result in 10 out of 11 swing states (with an average accuracy of 91%). Using this metric we penalize the inversions of polarity which can be a crucial issue while analyzing these kinds of states characterized by a high degree of uncertainty. Notice that, for what concerns North Carolina, neither the estimates achieved by IOM-NN nor the opinion polls were in-line with the actual outcome in this state. This is a common situation as the results achieved by the polls and IOM-NN must be understood as an estimate of the polarization of public opinion in the weeks preceding the Election Day, not always in accordance with the actual behavior of voters. Moreover, a noteworthy advantage of IOM-NN with respect to traditional opinion polls, is the ability to capture the opinion of a larger number of people more quickly and at a lower cost. This makes IOM-NN a valid support to enhance or even replace opinion polls, by providing relevant insights useful to understand the dynamics of the election campaign.
The goal of this last step is to model the political orientation of Twitter users from an emotional point of view. To this purpose, we used the SentiStrength tool (as explained earlier in Sect. 3.4), for discovering the existing relationships between user polarity and the sentiment expressed in referring to the two presidential candidates. Then, for each polarized tweet we explored the emotion the tweet conveys. Figures 13 and 14 describe the sentiment and the emotional state of the tweets with the relative intensity of the tweets produced by Trump and Biden supporters, respectively.
What appears evident is that, on average, the tweets produced by Trump’s supporters are significantly more positive than those produced by Biden’s supporters, which devote a significant number of negative tweets to their opponent.
For what concerns the detected emotions, Trump’s supporters express joy and confidence about Trump, while fear about Biden’s election. Biden’s supporters, instead, show trust and anticipation in having Biden as future president of the USA, with a more marked presence of negative emotions about Trump, like anger, disgust and sadness.
Tables 4 and 5 show various examples of tweets including in the analysis, showing how our approach can model social media conversation from an emotional point of view.