Characterizing the 2016 Russian IRA influence campaign

  • Adam BadawyEmail author
  • Aseel Addawood
  • Kristina Lerman
  • Emilio Ferrara
Original Article


Until recently, social media were seen to promote democratic discourse on social and political issues. However, this powerful communication ecosystem has come under scrutiny for allowing hostile actors to exploit online discussions in an attempt to manipulate public opinion. A case in point is the ongoing U.S. Congress investigation of Russian interference in the 2016 U.S. election campaign, with Russia accused of, among other things, using trolls (malicious accounts created for the purpose of manipulation) and bots (automated accounts) to spread propaganda and politically biased information. In this study, we explore the effects of this manipulation campaign, taking a closer look at users who re-shared the posts produced on Twitter by the Russian troll accounts publicly disclosed by U.S. Congress investigation. We collected a dataset of 13 million election-related posts shared on Twitter in the year of 2016 by over a million distinct users. This dataset includes accounts associated with the identified Russian trolls as well as users sharing posts in the same time period on a variety of topics around the 2016 elections. We use label propagation to infer the users’ ideology based on the news sources they share. We are able to classify a large number of the users as liberal or conservative with precision and recall above 84%. Conservative users who retweet Russian trolls produced significantly more tweets than liberal ones, about 8 times as many in terms of tweets. Additionally, trolls’ position in the retweet network is stable overtime, unlike users who retweet them who form the core of the election-related retweet network by the end of 2016. Using state-of-the-art bot detection techniques, we estimate that about 5% and 11% of liberal and conservative users are bots, respectively. Text analysis on the content shared by trolls reveal that conservative trolls talk about refugees, terrorism, and Islam, while liberal trolls talk more about school shootings and the police. Although an ideologically broad swath of Twitter users were exposed to Russian trolls in the period leading up to the 2016 U.S. Presidential election, it is mainly conservatives who help amplify their message.


Social media manipulation Russian trolls Bots Influence campaigns 



The authors gratefully acknowledge support by the Air Force Office of Scientific Research (Award #FA9550-17-1-0327). The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of AFOSR or the U.S. Government.


  1. Adamic LA, Glance N (2005) The political blogosphere and the 2004 US election: divided they blog. In: Proceedings of the 3rd international workshop on Link discoveryGoogle Scholar
  2. Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R (2011) Sentiment analysis of Twitter data. In: Proceedings of the workshop on languages in social media. Association for Computational Linguistics, pp 30–38Google Scholar
  3. Alarifi A, Alsaleh M, Al-Salman A (2016) Twitter turing test: identifying social machines. Inf Sci 372:332–346CrossRefGoogle Scholar
  4. Aral S, Walker D (2012) Identifying influential and susceptible members of social networks. Science 337(6092):337–341MathSciNetCrossRefGoogle Scholar
  5. Aral S, Muchnik L, Sundararajan A (2009) Distinguishing influence-based contagion from homophily-driven diffusion in dynamic networks. Proc Natl Acad Sci 106(51):21544–21549CrossRefGoogle Scholar
  6. Badawy A, Ferrara E, Lerman K (2018) Analyzing the digital traces of political manipulation: the 2016 Russian interference Twitter campaign. In: ASONAMGoogle Scholar
  7. Bakshy E, Hofman J, Mason W, Watts D (2011) Everyone’s an influencer: quantifying influence on Twitter. In: 4th WSDMGoogle Scholar
  8. Bakshy E, Messing S, Adamic LA (2015) Exposure to ideologically diverse news and opinion on Facebook. Science 348(6239):1130–1132MathSciNetCrossRefGoogle Scholar
  9. Barberá P, Wang N, Bonneau R, Jost JT, Nagler J, Tucker J, González-Bailón S (2015) The critical periphery in the growth of social protests. PLoS ONE 10(11):e0143611CrossRefGoogle Scholar
  10. Bekafigo MA, McBride A (2013) Who tweets about politics? Political participation of Twitter users during the 2011 gubernatorial elections. Soc Sci Comput Rev 31(5):625–643CrossRefGoogle Scholar
  11. Bessi A, Ferrara E (2016) Social bots distort the 2016 us presidential election online discussion. First Monday. CrossRefGoogle Scholar
  12. Bond RM, Fariss CJ, Jones JJ, Kramer AD, Marlow C, Settle JE, Fowler JH (2012) A 61-million-person experiment in social influence and political mobilization. Nature 489(7415):295CrossRefGoogle Scholar
  13. Bruns A, Burgess JE (2011) The use of Twitter hashtags in the formation of ad hoc publics. In: 6th ECPR general conferenceGoogle Scholar
  14. Buckels EE, Trapnell PD, Paulhus DL (2014) Trolls just want to have fun. Personal Individ Differ 67:97–102CrossRefGoogle Scholar
  15. Carlisle JE, Patton RC (2013) Is social media changing how we understand political engagement? An analysis of Facebook and the 2008 presidential election. Political Res Q 66(4):883–895CrossRefGoogle Scholar
  16. Centola D (2010) The spread of behavior in an online social network experiment. Science 329(5996):1194–1197CrossRefGoogle Scholar
  17. Centola D (2011) An experimental study of homophily in the adoption of health behavior. Science 334(6060):1269–1272CrossRefGoogle Scholar
  18. Conover M, Gonçalves B, Ratkiewicz J, Flammini A, Menczer F (2011a) Predicting the political alignment of Twitter users. In: Proceedings of 3rd IEEE conference on social computing, pp 192–199Google Scholar
  19. Conover M, Ratkiewicz J, Francisco MR, Gonçalves B, Menczer F, Flammini A (2011b) Political polarization on Twitter. ICWSM 133:89–96Google Scholar
  20. Conover MD, Davis C, Ferrara E, McKelvey K, Menczer F, Flammini A (2013a) The geospatial characteristics of a social movement communication network. PLoS ONE 8(3):e55957CrossRefGoogle Scholar
  21. Conover MD, Ferrara E, Menczer F, Flammini A (2013b) The digital evolution of occupy wall street. PLoS ONE 8(5):e64679CrossRefGoogle Scholar
  22. Csardi G, Nepusz T (2006) The igraph software package for complex network research. InterJournal Complex Syst 1695(5):1–9Google Scholar
  23. Davis CA, Varol O, Ferrara E, Flammini A, Menczer F (2016) Botornot: a system to evaluate social bots. In: Proceedings of 25th international conference on world wide web, pp 273–274Google Scholar
  24. Diakopoulos NA, Shamma DA (2010) Characterizing debate performance via aggregated Twitter sentiment. In: SIGCHI ConferenceGoogle Scholar
  25. DiGrazia J, McKelvey K, Bollen J, Rojas F (2013) More tweets, more votes: social media as a quantitative indicator of political behavior. PLoS ONE 8(11):e79449CrossRefGoogle Scholar
  26. Dutt R, Deb A, Ferrara E (2018) ‘Senator, we sell ads’: analysis of the 2016 Russian Facebook ads campaign. In: Third international conference on intelligent information technologies (ICIIT 2018)Google Scholar
  27. Effing R, Van Hillegersberg J, Huibers T(2011) Social media and political participation: are Facebook, Twitter and Youtube democratizing our political systems? In: Electronic participation, pp 25–35CrossRefGoogle Scholar
  28. El-Khalili S (2013) Social media as a government propaganda tool in post-revolutionary Egypt. First Monday. CrossRefGoogle Scholar
  29. Enli GS, Skogerbø E (2013) Personalized campaigns in party-centred politics: Twitter and Facebook as arenas for political communication. Inf Commun Soc 16(5):757–774CrossRefGoogle Scholar
  30. Ferrara E (2017) Disinformation and social bot operations in the run up to the 2017 French presidential election. First Monday. CrossRefGoogle Scholar
  31. Ferrara E (2018) Measuring social spam and the effect of bots on information diffusion in social media. In: Lehmann S, Ahn YY (eds) Complex spreading phenomena in social systems. Springer, Cham, pp 229–255CrossRefGoogle Scholar
  32. Ferrara E, Varol O, Davis C, Menczer F, Flammini A (2016a) The rise of social bots. Commun ACM 59(7):96–104CrossRefGoogle Scholar
  33. Ferrara E, Varol O, Menczer F, Flammini A (2016b) Detection of promoted social media campaigns. In: Tenth international AAAI conference on web and social media, pp 563–566Google Scholar
  34. Fourney A, Racz MZ, Ranade G, Mobius M, Horvitz E (2017) Geographic and temporal trends in fake news consumption during the 2016 US presidential election. In: CIKM, vol 17Google Scholar
  35. Freitas C, Benevenuto F, Ghosh S, Veloso A (2015) Reverse engineering socialbot infiltration strategies in Twitter. In: 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp 25–32Google Scholar
  36. Gibson RK, McAllister I (2006) Does cyber-campaigning win votes? Online communication in the 2004 australian election. J Elect Public Opin Parties 16(3):243–263CrossRefGoogle Scholar
  37. González-Bailón S, Borge-Holthoefer J, Rivero A, Moreno Y (2011) The dynamics of protest recruitment through an online network. Sci Rep 1:197CrossRefGoogle Scholar
  38. González-Bailón S, Borge-Holthoefer J, Moreno Y (2013) Broadcasters and hidden influentials in online protest diffusion. Am Behav Sci 57(7):943–965CrossRefGoogle Scholar
  39. Howard P (2006) New media campaigns and the managed citizen. Cambridge University Press, CambridgeGoogle Scholar
  40. Hwang T, Pearce I, Nanis M (2012) Socialbots: voices from the fronts. Interactions 19(2):38–45CrossRefGoogle Scholar
  41. Kloumann IM, Danforth CM, Harris KD, Bliss CA, Dodds PS (2012) Positivity of the english language. PLoS ONE 7(1):e29484CrossRefGoogle Scholar
  42. Kollanyi B, Howard PN, Woolley SC (2016) Bots and automation over Twitter during the first us presidential debate. Political BotsGoogle Scholar
  43. Kudugunta S, Ferrara E (2018) Deep neural networks for bot detection. Inf Sci 467(October):312–322CrossRefGoogle Scholar
  44. Loader BD, Mercea D (2011) Networking democracy? Social media innovations and participatory politics. Inf Commun Soc 14(6):757–769CrossRefGoogle Scholar
  45. Messias J, Schmidt L, Oliveira R, Benevenuto F (2013) You followed my bot! Transforming robots into influential users in Twitter. First Monday. CrossRefGoogle Scholar
  46. Metaxas PT, Mustafaraj E (2012) Social media and the elections. Science 338(6106):472–473CrossRefGoogle Scholar
  47. Monsted B, Sapiezynski P, Ferrara E, Lehmann S (2017) Evidence of complex contagion of information in social media: an experiment using Twitter bots. PLoS ONE 12(9):1–12CrossRefGoogle Scholar
  48. Pennycook G, Rand DG (2017) Assessing the effect of “disputed” warnings and source salience on perceptions of fake news accuracy. Social Science Research Network.
  49. Raghavan UN, Albert R, Kumara S (2007) Near linear time algorithm to detect community structures in large-scale networks. Phys Rev E 76(3):036106CrossRefGoogle Scholar
  50. Ratkiewicz J, Conover M, Meiss M, Gonçalves B, Patil S, Flammini A, Menczer F (2011a) Truthy: mapping the spread of astroturf in microblog streams. In: 20th WWW conference, pp 249–252Google Scholar
  51. Ratkiewicz J, Conover M, Meiss MR, Gonçalves B, Flammini A, Menczer F (2011b) Detecting and tracking political abuse in social media. In: ICWSM, vol 11, pp 297–304Google Scholar
  52. Savage S, Monroy-Hernandez A, öllerer TH (2016) Botivist: calling volunteers to action using online bots. In: 19th CSCWGoogle Scholar
  53. Shirky C (2011) The political power of social media: technology, the public sphere, and political change. Foreign Aff 90:28–41Google Scholar
  54. Shorey S, Howard PN (2016) Automation, algorithms, and politics: a research review. Int J Commun 10:5032–5055Google Scholar
  55. Stella M, Ferrara E, De Domenico M (2018) Bots increase exposure to negative and inflammatory content in online social systems. Proc Natl Acad Sci 115(49):12435–12440CrossRefGoogle Scholar
  56. Subrahmanian V, Azaria A, Durst S, Kagan V, Galstyan A, Lerman K, Zhu L, Ferrara E, Flammini A, Menczer F (2016) The DARPA Twitter bot challenge. Computer 49(6):38–46CrossRefGoogle Scholar
  57. Tufekci Z (2014) Big questions for social media big data: representativeness, validity and other methodological pitfalls. In: ICWSMGoogle Scholar
  58. Tufekci Z, Wilson C (2012) Social media and the decision to participate in political protest: observations from Tahrir Square. J Commun 62(2):363–379CrossRefGoogle Scholar
  59. Varol O, Ferrara E, Ogan CL, Menczer F, Flammini A (2014) Evolution of online user behavior during a social upheaval. In: Proceedings of the 2014 ACM conference on web scienceGoogle Scholar
  60. Varol O, Ferrara E, Davis C, Menczer F, Flammini A (2017a) Online human–bot interactions: detection, estimation, and characterization. In: ICWSM, pp 280–289Google Scholar
  61. Varol O, Ferrara E, Menczer F, Flammini A (2017b) Early detection of promoted campaigns on social media. EPJ Data Sci 6(13):13CrossRefGoogle Scholar
  62. Warriner AB, Kuperman V, Brysbaert M (2013) Norms of valence, arousal, and dominance for 13,915 english lemmas. Behav Res Methods 45(4):1191–1207CrossRefGoogle Scholar
  63. Wilson T, Wiebe J, Hoffmann P (2005) Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the conference on human language technology and empirical methods in natural language processing. Association for Computational Linguistics, pp 347–354Google Scholar
  64. Woolley SC, Howard PN (2016) Automation, algorithms, and politics: introduction. Int J Commun 10:9Google Scholar

Copyright information

© Springer-Verlag GmbH Austria, part of Springer Nature 2019

Authors and Affiliations

  • Adam Badawy
    • 1
    Email author
  • Aseel Addawood
    • 2
  • Kristina Lerman
    • 1
  • Emilio Ferrara
    • 1
  1. 1.USC Information Sciences InstituteLos AngelesUSA
  2. 2.University of Illinois at Urbana-ChampaignChampaignUSA

Personalised recommendations