Mining online communities to inform strategic messaging: practical methods to identify community-level insights

  • Matthew Benigni
  • Kenneth Joseph
  • Kathleen M. Carley


The ability of OSNs to propagate civil unrest has been powerfully observed through the rise of the ISIS and the ongoing conflict in Crimea. As a result, the ability to understand and in some cases mitigate the effects of user communities promoting civil unrest online has become an important area of research. Although methods to detect large online extremist communities have emerged in literature, the ability to summarize community content in meaningful ways remains an open research question. We introduce novel applications of the following methods: ideological user clustering with bipartite spectral graph partitioning, narrative mining with hash tag co-occurrence graph clustering, and identifying radicalization with directed URL sharing networks. In each case we describe how the method can be applied to social media. We subsequently apply them to online Twitter communities interested in the Syrian revolution and ongoing Crimean conflict.


Social networks Online extremist community Online extremism Social media Twitter Hashtags Terrorism ISIS Euromaidan 



This work was supported in part by the Office of Naval Research (ONR) through a MINERVA N000141310835 on State Stability. Additional support for this project was provided by the center for Computational Analysis of Social and Organizational Systems (CASOS) at CMU. The views ond conclu- sions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Office of Naval Research or the U.S. Government.


  1. Barbera P (2014) Tweeting the revolution: social media use and the #euromaidan protests | huffington postGoogle Scholar
  2. Benigni M, Carley KM (2016) From tweets to intelligence: understanding the Islamic Jihad supporting community on Twitter. In social computing, behavioral-cultural modeling, and prediction, page to appear. Springer, New YorkGoogle Scholar
  3. Benigni M, Joseph K, Carley KM (2017) Online extremism and the communities that sustain it: Detecting the isis supporting community on twitter. Under Revision to PLOS ONEGoogle Scholar
  4. Berger JM, Morgan J (2015) The isis twitter census: defining and describing the population of isis supporters on twitter. The Brookings Project on US Relations with the Islamic World 3(20)Google Scholar
  5. Berger JM (2015) Tailored online interventions: the Islamic state’s recruitment strategy. Combating Terrorism Center SentinelGoogle Scholar
  6. Blondel VD, Guillaume JL, Lambiotte R, Lefebvre E (2008) Fast unfolding of communities in large networks. J Stat Mech 2008(10):P10008 CrossRefGoogle Scholar
  7. Callimachi R (2015) ISIS and the Lonely Young American. The New York TimesGoogle Scholar
  8. Carley KM (2006) A dynamic network approach to the assessment of terrorist groups and the impact of alternative courses of action. Technical reportGoogle Scholar
  9. Chang HC (2010) A new perspective on twitter hashtag use: Diffusion of innovation theory. Proc Am Soc Inf Sci Technol 47(1):1–4Google Scholar
  10. DeMasi O, Mason D, Ma J (2016) Understanding communities via hashtag engagement: a clustering based approach. In: Tenth International AAAI Conference on Web and Social MediaGoogle Scholar
  11. Dewey T, Kaden J, Marks M, Matsushima S, Zhu B (2012) The impact of social media on social unrest in the Arab Spring. Int Policy ProgGoogle Scholar
  12. Dhillon IS (2001) Co-clustering documents and words using bipartite spectral graph partitioning. In: Proceedings of the seventh ACM SIGKDD international conference on knowledge discovery and data mining, ACM, pp 269–274Google Scholar
  13. Dozier K (2016) Anti-ISIS-Propaganda Czars Ninja war plan: we were never here, March 2016Google Scholar
  14. Fortunato S (2010) Community detection in graphs. Physics Rep 486(3):75–174CrossRefGoogle Scholar
  15. Glasgow K, Fink C (2013) Hashtag lifespan and social networks during the london riots. In: Greenberg AM, Kennedy WG, Bos ND (eds) Social computing, behavioral-cultural modeling and prediction. Springer, Berlin, pp 311–320. doi: 10.1007/978-3-642-37210-0_34 CrossRefGoogle Scholar
  16. Goodman LA (1961) Snowball sampling. Ann Math Stat 32(1):148–170CrossRefGoogle Scholar
  17. Hamilton WL, Clark K, Leskovec J, Jurafsky D (2016) Inducing domain-specific sentiment lexicons from unlabeled corporaGoogle Scholar
  18. Herrick D (2016) The social side of cyber power? social media and cyber operations. In: Cyber Conflict (CyCon), 2016 8th International Conference on, NATO CCD COE, pp 99–111Google Scholar
  19. Howard PN, Parks MR (2012) Social media and political change: capacity, constraint, and consequence. J Commun 62(2):359–362CrossRefGoogle Scholar
  20. Hussain MM, Howard PN (2013) What best explains successful protest cascades? ICTs and the fuzzy causes of the Arab Spring. Int Stud Rev 15(1):48–66CrossRefGoogle Scholar
  21. Jensen DN (2016) Lennart meri and the ’new normal’ | huffington postGoogle Scholar
  22. Johnson NF, Zheng M, Vorobyeva Y, Gabriel A, Qi H, Velasquez N, Manrique P, Johnson D, Restrepo E, Song C, Wuchty S (2016) online ecology of adversarial aggregates: ISIS and beyond. Science 352(6292):1459–1463CrossRefGoogle Scholar
  23. Juris JS (2012) Reflections on #occupy everywhere: social media, public space, and emerging logics of aggregation. Am Ethnol 39(2):259–279CrossRefGoogle Scholar
  24. Lloyd Stuart (1982) Least squares quantization in pcm. IEEE Trans Inf Theory 28(2):129–137CrossRefGoogle Scholar
  25. Loader BD, Mercea D (2011) Networking democracy? Inf Commun Soc 14:757–769CrossRefGoogle Scholar
  26. Lotan G, Graeff E, Ananny M, Gaffney D, Pearce I, Boyd D (2011) The revolutions were tweeted: information flows during the 2011 Tunisian and Egyptian revolutions. Int J Commun 5:1375–1405Google Scholar
  27. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, vol 1, University of California Press, pp 281–297Google Scholar
  28. Nanabhay M, Farmanfarmaian R (2011) From spectacle to spectacular: how physical space, social media and mainstream broadcast amplified the public sphere in Egypt’s Revolution’. J N Afr Stud 16(4):573–603CrossRefGoogle Scholar
  29. Papadopoulos S, Kompatsiaris Y, Vakali A, Spyridonos P (2012) Community detection in social media. Data Min Knowl Discov 24(3):515–554CrossRefGoogle Scholar
  30. Peel L, Larremore DB, Clauset A (2016) The ground truth about metadata and community detection in networksGoogle Scholar
  31. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. Proc Emprical Methods Nat Lang Process 12:1532–1543Google Scholar
  32. REST Twitter (2016) APIGoogle Scholar
  33. Roxburgh G (2016) Ukraine wins the 2016 eurovision song contest | news | eurovision song contestGoogle Scholar
  34. Sakaki T, Okazaki M, Matsuo Y (2016) Earthquake shakes twitter users: real-time event detection by social sensors. In: Proceedings of the 19th International Conference on World Wide Web WWW ’10, ACM, pp 851–860Google Scholar
  35. Settles B (2009) Active learning literature survey. Computer Sciences Technical Report 1648. University of Wisconsin, MadisonGoogle Scholar
  36. Shamanska A (2016) Hackers in Ukraine deface separatist websites to mark victory day. Accessed 23 Oct 2016Google Scholar
  37. Starbird K, Palen L (2012) (How) will the revolution be retweeted?: information diffusion and the 2011 Egyptian uprising. In: Proceedings of the acm 2012 conference on computer supported cooperative work, ACM, 2012, pp 7–16Google Scholar
  38. Steinbach M, Karypis G, Kumar V, et al. (2000) A comparison of document clustering techniques. In: KDD workshop on text mining, vol 400, Boston, pp 525–526Google Scholar
  39. Szostek J (2014) The media battles of Ukraine’s EuroMaidan. Digital Icons 11:1–19Google Scholar
  40. Tang L, Wang X, Liu H (2009) Uncoverning groups via heterogeneous interaction analysis. In: Ninth IEEE International Conference on Data Mining, ICDM’09, IEEE, pp 503–512Google Scholar
  41. Tufekci Z (2014) Big questions for social media big data: representativeness, validity and other methodological pitfalls. In: ICWSM ’14: Proceedings of the 8th international AAAI conference on weblogs and social mediaGoogle Scholar
  42. UNIAN (2016) Rada adopts bill allowing lutsenko to be nominated prosecutor general. Accessed 20 Oct 2016Google Scholar
  43. Veilleux-Lepage Y (2015) Paradigmatic Shifts in Jihadism in Cyberspace: the emerging role of unaffiliated sympathizers in the Islamic State’s social media strategy. J Terror ResGoogle Scholar
  44. Veilleux-Lepage Y (2014) Retweeting the Caliphate: the role of soft-sympathizers in the Islamic state’s social media strategy. In: 2014 6th International Terrorism and Transnational Crime Conference, March 2014Google Scholar
  45. Wagstaff K, Cardie C, Rogers S , Schrdl S, et al. (2001)Constrained k-means clustering with background knowledge. In: ICML, vol 1, pp 577–584Google Scholar
  46. Wasserman S, Faust K (1994) Social network analysis: methods and applications. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  47. Weber I, Garimella VRK, Teka A (2013) Political hashtag trends. In: Serdyukov P, Kuznetsov OS, Kamps J, Agichtein E, Segalovich I, Yilmaz E (eds) Advances in information retrieval. Springer, Berlin, pp 857–860. doi: 10.1007/978-3-642-36973-5_102 CrossRefGoogle Scholar
  48. Wei W, Joseph K, Liu H, Carley KM (2015) The fragility of twitter social networks against suspended users. In: Proceedings of the 2015 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), pp 9–16Google Scholar
  49. Wolfsfeld G, Segev E, Sheafer T (2013) Social media and the arab spring politics comes first. Int J Press 18(2):115–137CrossRefGoogle Scholar
  50. Wood P (2016) IS conflict: counting the civilian cost of US-led air strikesGoogle Scholar
  51. Zweig KA, Kaufmann M (2011) A systematic approach to the one-mode projection of bipartite graphs. Soc Netw Anal Min 1(3):187–218CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  1. 1.Center for Computational Analysis of Social and Organizational Systems (CASOS), Institute for Software ResearchCarnegie Mellon UniversityPittsburghUSA
  2. 2.Network Science InstituteNortheastern UniversityBostonUSA
  3. 3.Institute for Software ResearchCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations