Multimedia Tools and Applications

, Volume 78, Issue 3, pp 3553–3586 | Cite as

Social media signal detection using tweets volume, hashtag, and sentiment analysis

  • Faria Nazir
  • Mustansar Ali Ghazanfar
  • Muazzam Maqsood
  • Farhan Aadil
  • Seungmin Rho
  • Irfan MehmoodEmail author


Social Media is a well-known platform for users to create, share and check the new information. The world becomes a global village because of the utilization of internet and social media. The data present on Twitter contains information of great importance. There is a strong need to extract valuable information from this huge amount of data. A key research challenge in this area is to analyze and process this huge data and detect the signals or spikes. Existing work includes sentiment analysis for Twitter, hashtag analysis, and event detection but spikes/signal detection from Twitter remains an open research area. From this line of research, we propose a signal detection approach using sentiment analysis from Twitter data (tweets volume, top hashtag and sentiment analysis). In this paper, we propose three algorithms for signal detection in tweets volume, tweets sentiment and top hashtag. The algorithms are the- Average moving threshold algorithm, Gaussian algorithm, and hybrid algorithm. The hybrid algorithm is a combination of the average moving threshold algorithm and Gaussian algorithm. The proposed algorithms are tested over real-time data extracted from Twitter and two large publically available datasets- Saudi Aramco dataset and BP America dataset. Experimental results show that hybrid algorithm outperforms the Gaussian and average moving threshold algorithm and achieve a precision of 89% on real-time tweets data, 88% on Saudi Aramco dataset and 81% on BP America dataset with the recall of 100%.


Signal detection Sentiment analysis Social media analysis Twitter 



This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1A09919551).


  1. 1.
    Abdelhaq H, Sengstock C, Gertz M (2013) Eventweet: online localized event detection from twitter. Proceedings of the VLDB Endowment 6:1326–1329CrossRefGoogle Scholar
  2. 2.
    Adnan M, Longley P (2013) Analysis of twitter usage in london, paris, and new york city, in 16th AGILE international conference on geographic information science, Leuven, pp. 1–7Google Scholar
  3. 3.
    Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R (2011) Sentiment analysis of twitter data, in Proceedings of the workshop on languages in social media, pp. 30–38Google Scholar
  4. 4.
    Alag S (2008) Collective intelligence in action: Manning Publications Co.Google Scholar
  5. 5.
    Asur S, Huberman BA (2010) Predicting the future with social media, in Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology-Volume 01, pp. 492–499Google Scholar
  6. 6.
    Bagui S, Nguyen LT (2015) Database sharding: to provide fault tolerance and scalability of big data on the cloud. International Journal of Cloud Applications and Computing (IJCAC) 5:36–52CrossRefGoogle Scholar
  7. 7.
    Bastos MT (2015) Shares, pins, and tweets: news readership from daily papers to social media. Journal Stud 16:305–325Google Scholar
  8. 8.
    Chen L, Roy A (2009) Event detection from flickr data through wavelet-based spatial analysis, in Proceedings of the 18th ACM conference on Information and knowledge management, pp. 523–532Google Scholar
  9. 9.
    Ellison NB (2007) Social network sites: definition, history, and scholarship. J Comput-Mediat Commun 13:210–230CrossRefGoogle Scholar
  10. 10.
    Ellison N, Steinfield C, Lampe C (2006) Spatially bounded online social networks and social capital, International Communication Association, vol. 36Google Scholar
  11. 11.
    Ferragina P, Piccinno F, Santoro R (2015) On analyzing hashtags in twitter, in International Conference on Web and Social Media (ICWSM), pp. 110–119Google Scholar
  12. 12.
    Fung GPC, Yu JX, Yu PS, Lu H (2005) Parameter free bursty events detection in text streams, in Proceedings of the 31st international conference on Very large data bases, pp. 181–192Google Scholar
  13. 13.
    Gupta B, Agrawal DP, Yamaguchi S (2016) Handbook of research on modern cryptographic solutions for computer and cyber security: IGI GlobalGoogle Scholar
  14. 14.
    He Q, Chang K, Lim E-P (2007) Analyzing feature trajectories for event detection, in Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pp. 207–214Google Scholar
  15. 15.
    Herbert A, Nenadic G (2015) Text Mining Twitter Posts For Signs of Social IsolationGoogle Scholar
  16. 16.
    Kaplan AM, Haenlein M (2010) Users of the world, unite! The challenges and opportunities of social media. Business horizons 53:59–68CrossRefGoogle Scholar
  17. 17.
    Kleinberg J (2003) Bursty and hierarchical structure in streams. Data Min Knowl Disc 7:373–397MathSciNetCrossRefGoogle Scholar
  18. 18.
    Kouloumpis E, Wilson T, Moore JD (2011) Twitter sentiment analysis: the good the bad and the omg! Icwsm 11:164Google Scholar
  19. 19.
    Lazer D, Pentland AS, Adamic L, Aral S, Barabasi AL, Brewer D et al (2009) Life in the network: the coming age of computational social science. Science (New York, NY) 323:721CrossRefGoogle Scholar
  20. 20.
    Li Y, Wang G, Nie L, Wang Q, Tan W (2018) Distance metric optimization driven convolutional neural network for age invariant face recognition. Pattern Recogn 75:51–62CrossRefGoogle Scholar
  21. 21.
    McCormick TH, Lee H, Cesare N, Shojaie A, Spiro ES (2017) Using twitter for demographic and social science research: tools for data collection and processing. Sociol Methods Res 46:390–421MathSciNetCrossRefGoogle Scholar
  22. 22.
    Mcnee SM (2006) Meeting user information needs in recommender systems: University of MinnesotaGoogle Scholar
  23. 23.
    Melville P, Sindhwani V, Lawrence R (2009) Social media analytics: channeling the power of the blogosphere for marketing insight. Proc of the WIN 1:1–5Google Scholar
  24. 24.
    Murphy J, Link MW, Childs JH, Tesfaye CL, Dean E, Stern M et al (2014) Social media in public opinion research: executive summary of the Aapor task force on emerging technologies in public opinion research. Public Opinion Quarterly 78:788–794CrossRefGoogle Scholar
  25. 25.
    O'Connor B, Balasubramanyan R, Routledge BR, Smith NA (2010) From tweets to polls: linking text sentiment to public opinion time series. Icwsm 11:1–2Google Scholar
  26. 26.
    Ouf S, Nasr M (2015) Cloud computing: the future of big data management. International Journal of Cloud Applications and Computing (IJCAC) 5:53–61CrossRefGoogle Scholar
  27. 27.
    Petrović S, Osborne M, Lavrenko V (2010) Streaming first story detection with application to twitter, in Human language technologies: The 2010 annual conference of the north american chapter of the association for computational linguistics, pp. 181–189Google Scholar
  28. 28.
    Petrovic S, Osborne M, McCreadie R, Macdonald C, Ounis I, Shrimpton L (2013) Can twitter replace newswire for breaking news?, in ICWSM Google Scholar
  29. 29.
    Qualman E (2010) Socialnomics: How social media transforms the way we live and do business: John Wiley & SonsGoogle Scholar
  30. 30.
    Ramasubramanian C, Ramya R (2013) Effective pre-processing activities in text mining using improved porter’s stemming algorithm. International Journal of Advanced Research in Computer and Communication Engineering 2:2278–1021Google Scholar
  31. 31.
    Sakaki T, Okazaki M, Matsuo Y (2010) Earthquake shakes Twitter users: real-time event detection by social sensors, in Proceedings of the 19th international conference on World wide web, pp. 851–860Google Scholar
  32. 32.
    Small TA (2011) What the hashtag? A content analysis of Canadian politics on twitter. Inf Commun Soc 14:872–895CrossRefGoogle Scholar
  33. 33.
    Spencer J, Uchyigit G (2012) Sentimentor: Sentiment analysis of twitter data, in Proceedings of European conference on machine learning and principles and practice of knowledge discovery in databases, pp. 56–66Google Scholar
  34. 34.
    Trusov M, Bucklin RE, Pauwels K (2009) Effects of word-of-mouth versus traditional marketing: findings from an internet social networking site. J Mark 73:90–102CrossRefGoogle Scholar
  35. 35.
    Wang X, Wei F, Liu X, Zhou M, Zhang M (2011) Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach, in Proceedings of the 20th ACM international conference on Information and knowledge management, pp. 1031–1040Google Scholar
  36. 36.
    Watts DJ (2004) The “new” science of networks. Annu Rev Sociol 30:243–270CrossRefGoogle Scholar
  37. 37.
    Weng J, Lee B-S (2011) Event detection in twitter. ICWSM 11:401–408Google Scholar
  38. 38.
    Yang Y, Pierce T, Carbonell J (1998) A study of retrospective and on-line event detection, in Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pp. 28–36Google Scholar
  39. 39.
    Zhang Z, Gupta BB (2016) Social media security and trustworthiness: overview and new direction, Future Generation Computer Systems Google Scholar
  40. 40.
    Zhang Z, Sun R, Zhao C, Wang J, Chang CK, Gupta BB (2017) CyVOD: a novel trinity multimedia social network scheme. Multimedia Tools and Applications 76:18513–18529CrossRefGoogle Scholar
  41. 41.
    Zheng L, Wang H, Gao S (2018) Sentimental feature selection for sentiment analysis of Chinese online reviews. Int J Mach Learn Cybern 9:75–84CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Software EngineeringUniversity of Engineering and Technology TaxilaTaxilaPakistan
  2. 2.Department of Computer ScienceCOMSATS University Islamabad, Attock CampusAttockPakistan
  3. 3.Department of Media SoftwareSungkyul UniversityAnyangSouth Korea
  4. 4.Department of SoftwareSejong UniversitySeoulSouth Korea

Personalised recommendations