Cognitive Computing Approaches for Human Activity Recognition from Tweets—A Case Study of Twitter Marketing Campaign

  • Jari JussilaEmail author
  • Prashanth Madhala
Conference paper
Part of the Springer Proceedings in Complexity book series (SPCOM)


Big data analytics is a growing area of research and has in the recent years introduced many applications for business sector to create value from data. Cognitive computing provides new approaches for analysing social media data, e.g. text analytics and machine vision, available via APIs and software libraries. This study focuses on cognitive computing approaches for recognizing and categorizing valence, arousal and human activity from tweets. The aim is to identify human activities from tweets and map these activities on pleasure and arousal dimensions using cognitive computing services in order to derive new insights for business. A case study of Twitter marketing campaign of a company providing stress management services is presented. In the marketing campaign participants submitted images and text on events that help them to recover from stress. These tweets are analysed with cognitive computing services to recognize activities that participants of the campaign performed to recover from stress and these are compared against qualitative analysis. The caption recognition was successful at detecting some of the activities of the users and overall results indicate that many participants cope with stress by e.g. performing outdoor activities. The two sentiment detection algorithms were also successful in detecting consumer sentiments from the text component of the tweets. Finally, implications for business are drawn from the results of analysis.


  1. 1.
    Jalonen, H., Jussila, J.: Developing a Conceptual Model for the Relationship Between Social Media Behavior, Negative Consumer Emotions and Brand Disloyalty. Presented at the (2016)Google Scholar
  2. 2.
    Magids, S., Zorfas, A., Leemon, D.: The new science of customer emotions. Harv. Bus. Rev. November 2, (2015)Google Scholar
  3. 3.
    Laros, F., Steenkamp, J.: Emotions in consumer behavior: a hierarchical approach. J. Bus. Res. 58, 1437–1445 (2005)CrossRefGoogle Scholar
  4. 4.
    Zhu, Z., Blanke, U., Calatroni, A., Tröster, G.: Prior knowledge of human activities from social data. In: Proceedings of the 2013 International Symposium on Wearable Computers. pp. 141–142. ACM (2013)Google Scholar
  5. 5.
    Jussila, J.: Social media in business-to-business companies’ innovation. (2015)
  6. 6.
    Madhala, P., Jussila, J., Aramo-Immonen, H., Suominen, A.: Systematic literature review on customer emotions in social media. In: ECSM 2018 5th European Conference on Social Media (2018)Google Scholar
  7. 7.
    Jussila, J., Boedeker, M., Jalonen, H., Helander, N.: Social media analytics empowering customer experience insight. In: Kavoura, A., Sakas, D., Tomaras, P. (eds.) Strategic Innovative Marketing, pp. 25–30. Springer, Cham (2017)CrossRefGoogle Scholar
  8. 8.
    Fosso Wamba, S., Akter, S., Edwards, A., Chopin, G., Gnanzou, D.: How “big data” can make big impact: Findings from a systematic review and a longitudinal case study. Int. J. Prod. Econ. 165, 234–246 (2015)CrossRefGoogle Scholar
  9. 9.
    Vatrapu, R.: Understanding social business. In: Emerging Dimensions of Technology Management. pp. 147–158. Springer India, India (2013)Google Scholar
  10. 10.
    Jussila, J., Menon, K., Gupta, J., Kärkkäinen, H.: Who is who in big social data? A bibliographic network analysis study. In: Proceedings of The 4th European Conference on Social Media. pp. 161–169. Academic Conferences and publishing limited, Vilnius (2017)Google Scholar
  11. 11.
    Coté, M.: The materiality of big social data. Cult. Stud. Rev. 20, 121–149 (2014)CrossRefGoogle Scholar
  12. 12.
    Bravo-Marquez, F., Mendoza, M., Poblete, B.: Meta-level sentiment models for big social data analysis. Knowl.-Based Syst. 69, 86–99 (2014)CrossRefGoogle Scholar
  13. 13.
    Batrinca, B., Treleaven, P.C.: Social media analytics: a survey of techniques, tools and platforms. AI Soc. 30, 89–116 (2015)CrossRefGoogle Scholar
  14. 14.
    Bello-Orgaz, G., Jung, J., Camacho, D.: Social big data: Recent achievements and new challenges. Inf. Fusion. 28, 44–59 (2016)CrossRefGoogle Scholar
  15. 15.
    Valiant, L.: Cognitive computation. In: Proceedings of IEEE 36th Annual Foundations of Computer Science. IEEE (1995)Google Scholar
  16. 16.
    Gutierrez-Garcia, J., López-Neri, E.: Cognitive computing: a brief survey and open research challenges. In: 3rd International Conference on Applied Computing and Information Technology/2nd International Conference on Computational Science and Intelligence. pp. 328–333. IEEE (2015)Google Scholar
  17. 17.
    Boyd, D., Crawford, K.: Critical Questions for Big Data. Information, Commun. Soc. 15, 662–679 (2012)Google Scholar
  18. 18.
    Bruns, A.: Faster than the speed of print: reconciling ’big data’social media analysis and academic scholarship. First Monday. 18, (2013)Google Scholar
  19. 19.
    Davenport, T.H., Prusak, L.: Working knowledge : how organizations manage what they know. Harvard Business School Press (1998)Google Scholar
  20. 20.
    Buyya, R., Yeo, C., Venugopal, S., Broberg, J., Brandic, I.: Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Futur. Gener. Comput. Syst. 25, 599–616 (2009)CrossRefGoogle Scholar
  21. 21.
    Ekman, P.: An argument for basic emotions. Cogn. Emot. 6, 169–200 (1992)CrossRefGoogle Scholar
  22. 22.
    Plutchik, R.: Robert: the nature of emotions. Am. Sci. 89, 344 (2001)CrossRefGoogle Scholar
  23. 23.
    Mehrabian, A., Russell, J.: An Approach to Environmental Psychology. The MIT Press, Cambridge (1974)Google Scholar
  24. 24.
    Seo, M., Barrett, L., Jin, S.: The structure of affect: history, theory, and implications for emotion research in organizations. In: Ashkanasy, N., Cooper, C. (eds.) New Horizons in Management. Research Companion to Emotion in Organizations. pp. 17–44. Edward Elgar Publishing, Northampton (2008)Google Scholar
  25. 25.
    Scherer, K.R.: What are emotions? And how can they be measured? Soc. Sci. Inf. 44, 695–729 (2005)CrossRefGoogle Scholar
  26. 26.
    Zhao, J., Gou, L., Wang, F., Zhou, M.: PEARL: an interactive visual analytic tool for understanding personal emotion style derived from social media. In: 2014 IEEE Conference on Visual Analytics Science and Technology (VAST). pp. 203–212. IEEE (2014)Google Scholar
  27. 27.
    Bradley, M., Lang, P.: Affective norms for English words (ANEW): Instruction manual and affective ratings. (1999)Google Scholar
  28. 28.
    Mehrabian, A.: Basic dimensions for a general psychological theory : implications for personality, social, environmental, and developmental studies. Oelgeschlager, Gunn & Hain (1980)Google Scholar
  29. 29.
    Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)CrossRefGoogle Scholar
  30. 30.
    Davidson, R.J., Sharon Begley, W.: The emotional life of your brain: how its unique patterns affect the way you think, feel, and live-and how you can change them (2012)Google Scholar
  31. 31.
    Zimmerman, C., Stein, M.-K., Hardt, D., Vatrapu, R.: Emergence of things felt: harnessing the semantic space of Facebook feeling tags (2015)Google Scholar
  32. 32.
    Russell, J.: Pancultural aspects of the human conceptual organization of emotions.–14108-001
  33. 33.
    Wang, Z., Chong, C.S., Lan, L., Yang, Y., Beng Ho, S., Tong, J.C.: Fine-grained sentiment analysis of social media with emotion sensing. In: 2016 Future Technologies Conference (FTC). pp. 1361–1364. IEEE (2016)Google Scholar
  34. 34.
    Socher, R., Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. (2013)Google Scholar
  35. 35.
    Chafale, D., Pimpalkar, A.: Review on developing corpora for sentiment analysis using Plutchik’s wheel of emotions with fuzzy logic. Int. J. Comput. Sci. Eng. (2014)Google Scholar
  36. 36.
    Menon, K., Karkkainen, H., Jussila, J., Huhtamaki, J., Mukkamala, R., Lasrado, L., Hussain, A.: Analysing the role of crowdfunding in entrepreneurial ecosystems: a social media event study of two competing product launches. Int. J. Entrep. Small Bus. 33, 575–606 (2018)CrossRefGoogle Scholar
  37. 37.
    Lavidge, R., Steiner, G.: A model for predictive measurements of advertising effectiveness. J. Mark. 59–62 (1961)Google Scholar
  38. 38.
    Larsen, H.H., Forsberg, J.M., Hemstad, S.V., Mukkamala, R.R., Hussain, A., Vatrapu, R.: TV ratings versus social media engagement: big social data analytics of the Scandinavian TV talk show Skavlan. In: 2016 IEEE International Conference on Big Data (Big Data). pp. 3849–3858. IEEE (2016)Google Scholar
  39. 39.
    Sarakit, P., Theeramunkong, T., Haruechaiyasak, C., Okumura, M.: Classifying emotion in Thai youtube comments. In: 2015 6th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES). pp. 1–5. IEEE (2015)Google Scholar
  40. 40.
    Li, W., Xu, H.: Text-based emotion classification using emotion cause extraction. Expert Syst. Appl. 41, 1742–1749 (2014)CrossRefGoogle Scholar
  41. 41.
    Shukri, S.E., Yaghi, R.I., Aljarah, I., Alsawalqah, H.: Twitter sentiment analysis: a case study in the automotive industry. In: 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT). pp. 1–5. IEEE (2015)Google Scholar
  42. 42.
    Wilson, T., Wiebe, J., Hoffmann, P.: Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing—HLT ’05. pp. 347–354. Association for Computational Linguistics, Morristown, NJ, USA (2005)Google Scholar
  43. 43.
    Strapparava, C., Valitutti, A.: WordNet-Affect: an affective extension of WordNet. In: Proceedings of the International Conference on Language Resources and Evaluation (2004)Google Scholar
  44. 44.
    Sun, X., Zhang, C., Li, G., Sun, D., Ren, F., Zomaya, A., Ranjan, R.: Detecting users’ anomalous emotion using social media for business intelligence. J. Comput. Sci. (2017)Google Scholar
  45. 45.
    Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems—CHI ’17, pp. 3506–3510. ACM Press, New York, USA (2017)Google Scholar
  46. 46.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Vanderplas, J.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)Google Scholar
  47. 47.
    Kristina, S., Shelley, K.: Developing the American time use survey activity classification system. Mon. Labor Rev. 3, (2005)Google Scholar
  48. 48.
    Zhu, Z., Blanke, U., Calatroni, A., Tröster, G.: Human activity recognition using social media data. In: Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia. p. 21 (2013)Google Scholar
  49. 49.
    Dearman, D., Sohn, T., Truong, K.: Opportunities exist: continuous discovery of places to perform activities. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 2429–2438. ACM (2011)Google Scholar
  50. 50.
    Beber, M.: Individual and group activity recognition in moving object trajectories (2017)Google Scholar
  51. 51.
    Saunders, M.N.K., Lewis, P., Thornhill, A.: Research methods for business students. Prentice Hall (2009)Google Scholar
  52. 52.
    Yin, R.K.: Case study research : design and methods. Sage Publications (2003)Google Scholar
  53. 53.
    Cowley, B.U., Torniainen, J.: A short review and primer on electrodermal activity in human computer interaction applications (2016)Google Scholar
  54. 54.
    Bradley, M., Lang, P.: Emotion and motivation (2000)Google Scholar
  55. 55.
    Jussila, J., Venho, N., Salonius, H., Moilanen, J., Liukkonen, J., Rinnetmäki, M.: Towards ecosystem for research and development of electrodermal activity applications. In: Proceedings of the 22nd International Academic Mindtrek Conference. pp. 79–87. ACM (2018)Google Scholar
  56. 56.
    Thelwall, M.: The Heart and soul of the web? Sentiment strength detection in the social web with SentiStrength. In: Holyst, J. (ed.) Cyberemotions. pp. 119–134. Springer (2017)Google Scholar
  57. 57.
    Hutto, C.J., Gilbert, E.: Vader: a parsimonious rule-based model for sentiment analysis of social media text. ICWSM. (2014)Google Scholar
  58. 58.
    Oinas-Kukkonen, H.: Personalization Myopia: a viewpoint to true personalization of information systems. In: Proceedings of the 22nd International Academic Mindtrek Conference. pp. 88–91. ACM (2018)Google Scholar
  59. 59.
    Bernabé-Moreno, J., Tejeda-Lorente, A., Porcel, C., Fujita, H., Herrera-Viedma, E.: Emotional profiling of locations based on social media. Procedia Comput. Sci. 55, 960–969 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Häme University of Applied SciencesHämeenlinnaFinland

Personalised recommendations