Skip to main content

Supporting customer-oriented marketing with artificial intelligence: automatically quantifying customer needs from social media

Abstract

The elicitation and monitoring of customer needs is an important task for businesses, allowing them to design customer-centric products and services and control marketing activities. While there are different approaches available, most lack in automation, scalability and monitoring capabilities. In this work, we demonstrate the feasibility towards an automated prioritization and quantification of customer needs from social media data. To do so, we apply a supervised machine learning approach on the example of previously labeled Twitter data from the domain of e-mobility. We descriptively code over 1000 German tweets and build eight distinct classification models, so that a resulting artifact can independently determine the probabilities of a tweet containing each of the eight previously defined needs. To increase the scope of application, we deploy the machine learning models as part of a web service for public use. The resulting artifact can provide valuable insights for need elicitation and monitoring when analyzing user-generated content on a large scale.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Notes

  1. 1.

    e-tankstelle, eauto, elektroauto, elektrofahrzeug, elektromobilitaet, elektromobilität, ladesaeule, ladesäule

  2. 2.

    ecar, electric mobility, EV vehicle, e-mobility, emobility

  3. 3.

    bmw i3, egolf, eup, fortwo electric drive, miev, nissan leaf, opel ampera, peugeot ion, renault zoe, tesla model s

  4. 4.

    In case one tweet contains multiple needs that belong to the same need category, only one instance is used in the implementation. Therefore, the effective frequencies of tweets containing need categories is slightly different than the total amounts shown in Table 3.

References

  1. Andrews, N. O., & Fox, E. A. (2007). Recent developments in document clustering.

    Google Scholar 

  2. Balakrishnan, V., & Lloyd-Yemoh, E. (2014). Stemming and lemmatization: A comparison of retrieval performances. Lecture Notes on Software Engineering, 2(3), 262.

    Google Scholar 

  3. Barberá, P., & Rivero, G. (2015). Understanding the political representativeness of twitter users. Social Science Computer Review, 33(6), 712–729.

    Google Scholar 

  4. Bergstra, J. S., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. In Advances in neural information processing systems (pp. 2546–2554).

    Google Scholar 

  5. Bermingham, A., & Smeaton, A. F. (2011). On using twitter to monitor political sentiment and predict election results.

    Google Scholar 

  6. Bifet, A., & Frank, E. (2010). Sentiment knowledge discovery in twitter streaming data. In B. Pfahringer, G. Holmes, & A. Hoffmann (Eds.), Discovery science (Vol. 6332, pp. 1–15). Berlin Heidelberg: Springer.

    Google Scholar 

  7. Bird, S., Klein, E., & Loper, E. (2009). Natural language processing with Python: Analyzing text with the natural language toolkit. “ O’Reilly Media, Inc.”

  8. Blindheim, J., Wulvik, A., & Steinert, M. (2016). Using secondary video material for user observation in the needfinding process for new product development and design. In Proceedings of the DESIGN 2016 14th International Design Conference.

  9. Bradley, A. P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7), 1145–1159.

    Google Scholar 

  10. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

    Google Scholar 

  11. Cavnar, W. B., & Trenkle, J. M. (1994). N-gram-based text categorization. Ann Arbor MI, 48113(2), 161–175.

    Google Scholar 

  12. Cawley, G. C., & Talbot, N. L. C. (2010). On over-fitting in model selection and subsequent selection Bias in performance evaluation. Journal of Machine Learning Research, 11, 2079–2107 Retrieved from http://jmlr.csail.mit.edu/papers/v11/cawley10a.html%5Cn, http://www.jmlr.org/papers/volume11/cawley10a/cawley10a.pdf. Accessed 2019-02-01.

    Google Scholar 

  13. Chawla, N. V. (2005). Data mining for imbalanced datasets: An overview. In Data mining and knowledge discovery handbook (pp. 853–867). Springer.

  14. Chen, R., & Lazer, M. (2013). Sentiment analysis of twitter feeds for the prediction of stock market movement. Stanford. Edu. Retrieved January, 25, 2013.

  15. Chen, H., & Zimbra, D. (2010). AI and opinion mining. IEEE Intelligent Systems, 25(3), 74–80.

    Google Scholar 

  16. Christopher, M., Payne, A., & Ballantyne, D. (1991). Relationship marketing: Bringing quality customer service and marketing together.

    Google Scholar 

  17. Cieliebak, M., Deriu, J., Egger, D., & Uzdilli, F. (2017). A twitter Corpus and benchmark resources for German sentiment analysis. Fifth International Workshop on Natural Language Processing for Social Media, (April), 45–51. https://doi.org/10.18653/v1/W17-1106

  18. Correa, T., Hinsley, A. W., & de Zúñiga, H. G. (2010). Who interacts on the web?: The intersection of users’ personality and social media use. Computers in Human Behavior, 26(2), 247–253. https://doi.org/10.1016/j.chb.2009.09.003 .

    Article  Google Scholar 

  19. Cuthbertson, R., Furseth, P. I., & Ezell, S. J. (2015). Innovating in a service-driven economy: Insights, application, and practice. Palgrave Macmillan UK Retrieved from https://books.google.de/books?id=kXWkCgAAQBAJ. Accessed 2019-02-01.

  20. Edvardsson, B., Kristensson, P., Magnusson, P., & Sundström, E. (2012). Customer integration within service development - a review of methods and an analysis of insitu and exsitu contributions. Technovation, 32, 419–429. https://doi.org/10.1016/j.technovation.2011.04.006 .

    Article  Google Scholar 

  21. Ester, M., Kriegel, H.-P., Sander, J., & Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining (Vol. 96, pp. 226–231).

    Google Scholar 

  22. Finin, T., Murnane, W., Karandikar, A., Keller, N., Martineau, J., & Dredze, M. (2010). Annotating named entities in twitter data with crowdsourcing. Proceedings of the NAACL HLT 2010 workshop on creating speech and language data with Amazon’s mechanical Turk.

  23. Garimella, K., Weber, I., & De Choudhury, M. (2016). Quote rts on twitter: Usage of the new feature for political discourse. In Proceedings of the 8th ACM conference on web science (pp. 200–204).

  24. Gerber, M. S. (2014). Predicting crime using twitter and kernel density estimation. Decision Support Systems, 61, 115–125.

    Google Scholar 

  25. Gollapudi, S. (2016). Practical machine learning. Packt Publishing. Retrieved from https://books.google.de/books?id=WmsdDAAAQBAJ. Accessed 2019-02-01.

  26. Gregor, S., & Hevner, A. R. (2013). Positioning and presenting design science research for Maximim impact. MIS Quarterly, 37(2), 337–355. https://doi.org/10.2753/MIS0742-1222240302 .

    Article  Google Scholar 

  27. Grinberg, M. (2018). Flask web development: Developing web applications with python. O’Reilly Media, Inc.

  28. Haas, D., Wang, J., Wu, E., & Franklin, M. J. (2015). Clamshell: Speeding up crowds for low-latency data labeling. Proceedings of the VLDB Endowment, 9(4), 372–383.

    Google Scholar 

  29. Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating ( ROC ) Curvel characteristic. Radiology, 143(1), 29–36. https://doi.org/10.1148/radiology.143.1.7063747 .

    Article  Google Scholar 

  30. Harding, J. A., Popplewell, K., Fung, R. Y. K., & Omar, A. R. (2001). Intelligent information framework relating customer requirements and product characteristics. Computers in Industry, 44(1), 51–65. https://doi.org/10.1016/S0166-3615(00)00074-9 .

    Article  Google Scholar 

  31. Hauser, J. R., & Griffin, A. (1993). The voice of the customer. Marketing Science, 12, 1–27.

    Google Scholar 

  32. Hirt, R., Kühl, N., & Satzger, G. (2017). An end-to-end process model for supervised machine learning classification: from problem to deployment in information systems. In Designing the Digital Transformation: DESRIST 2017 Research in Progress Proceedings of the 12th International Conference on Design Science Research in Information Systems and Technology. Karlsruhe, Germany. 30 May-1 Jun.

  33. Hsu, C.-W., Chang, C.-C., & Lin, C.-J. (2003). A practical guide to support vector classification.

  34. Hu, M., & Liu, B. (2004a). Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining (pp. 168–177).

  35. Hu, M., & Liu, B. (2004b). Mining opinion features in customer reviews. 19th National Conference on Artifical intelligence, 755–760. https://doi.org/10.1145/1014052.1014073

  36. Hu, M., & Liu, B. (2006). Opinion feature extraction using class sequential rules. In Proceedings of the AAAI spring symposium: Computational approaches to analyzing weblogs (pp. 61–66).

    Google Scholar 

  37. Hull, E., Jackson, K., & Dick, J. (2010). Requirements engineering. Springer Science & Business Media.

  38. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning. New York: Springer. Retrieved from https://link.springer.com/content/pdf/10.1007%2F978-1-4614-7138-7.pdf. Accessed 2019-02-01.

  39. Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260.

    Google Scholar 

  40. Jovic, A., Brkic, K., & Bogunovic, N. (2014). An overview of free software tools for general data mining. In Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2014 37th International Convention on (pp. 1112–1117).

  41. Kalchbrenner, N., Grefenstette, E., & Blunsom, P. (2014). A convolutional neural network for modelling sentences. In Proceedings of the 52nd annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.3115/v1/P14-1062

  42. Kaplan, A. M., & Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of social media. Business Horizons, 53(1), 59–68. https://doi.org/10.1016/j.bushor.2009.09.003 .

    Article  Google Scholar 

  43. Khalid, H. M., & Helander, M. G. (2006). Customer emotional needs in product design. Concurrent Engineering, 14(3), 197–206.

    Google Scholar 

  44. Kietzmann, J. H., Hermkens, K., McCarthy, I. P., & Silvestre, B. S. (2011). Social media? Get serious! Understanding the functional building blocks of social media. Business Horizons, 54(3), 241–251.

    Google Scholar 

  45. Kotler, P., & Armstrong, G. (2001). Principles of marketing. World wide web internet and web. Information Systems, 42, 105. https://doi.org/10.2307/1250103 .

    Article  Google Scholar 

  46. Kotsiantis, S. B. (2007). Supervised machine learning: A review of classification techniques. Informatica, 31, 249–268. https://doi.org/10.1115/1.1559160 .

    Article  Google Scholar 

  47. Kuechler, W., & Vaishnavi, V. (2012). A framework for theory development in design science research: Multiple perspectives. Journal of the Association for Information Systems. https://doi.org/10.1201/b18448-6 .

  48. Kuehl, N., Scheurenbrand, J., & Satzger, G. (2016). “Needmining: Identifying micro blog data containing customer needs”. Research Papers. 185. https://aisel.aisnet.org/ecis2016_rp/185.

  49. Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. In Applied predictive modeling. https://doi.org/10.1007/978-1-4614-6849-3 .

    Chapter  Google Scholar 

  50. Kühl, N., Goutier, M., Ensslen, A., & Jochem, P. (2018). Literature vs. Twitter: Empirical insights on customer needs in e-mobility. Journal of Cleaner Production. https://doi.org/10.1016/j.jclepro.2018.12.003.

  51. Kvaløy, O., Nieken, P., & Schöttner, A. (2015). Hidden benefits of reward: A field experiment on motivation and monetary incentives. European Economic Review, 76, 188–199.

    Google Scholar 

  52. Lee, D., Jeong, O.-R., & Lee, S.-G. (2008). Opinion mining of customer feedback data on the web. Proceedings of the 2nd international conference on ubiquitous information management and communication ICUIMC 08, 230. https://doi.org/10.1145/1352793.1352842

  53. Lemaître, G., Nogueira, F., & Aridas, C. K. (2017). Imbalanced-learn: A Python toolbox to tackle the curse of imbalanced datasets in machine learning. Journal of Machine Learning Research, 18(17), 1–5.

    Google Scholar 

  54. Limehouse, D. (1999). Know your customer. Work Study, 48(3), 100–102. https://doi.org/10.1108/00438029910262518 .

    Article  Google Scholar 

  55. Ling, C. X., Huang, J., & Zhang, H. (2003). AUC: A statistically consistent and more discriminating measure than accuracy. In IJCAI international joint conference on artificial intelligence (pp. 519–524).

  56. Loria, S. (2017). TextBlob.

  57. March, S. T., & Smith, G. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266 Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=epref&AN=DSS.AE.BEA.MARCH.DNSRIT&site=ehost-live. Accessed 2019-02-01.

    Google Scholar 

  58. Marshall, T. C., Lefringhausen, K., & Ferenczi, N. (2015). The big five, self-esteem, and narcissism as predictors of the topics people write about in Facebook status updates. Personality and Individual Differences, 85, 35–40. https://doi.org/10.1016/j.paid.2015.04.039 .

    Article  Google Scholar 

  59. Maynard, D., Bontcheva, K., & Rout, D. (2012). Challenges in developing opinion mining tools for social media. Proceedings of the@ NLP can u tag# Usergeneratedcontent, 15–22.

  60. Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013a). Efficient estimation of word representations in vector space, 1–12.

  61. Mikolov, T., Yih, W., & Zweig, G. (2013b). Linguistic regularities in continuous space word representations. In Proceedings of the annual conference of the north American chapter of the Association for Computational Linguistics: Human language technologies (HLT-NAACL) (Vol. 13, pp. 746–751).

  62. Misopoulos, F., Mitic, M., Kapoulas, A., & Karapiperis, C. (2014). Uncovering customer service experiences with twitter: The case of airline industry. Management Decision, 52(4), 705–723.

    Google Scholar 

  63. Mohri, M., Rostamizadeh, A., & Talwalkar, A. (2012). Foundations of machine learning. MIT Press.

  64. Montgomery, D. C. (2013). Design and Analysis of Experiments (Eighth Edi). Hoboken: Wiley.

    Google Scholar 

  65. Müller, V. C., & Bostrom, N. (2016). Future progress in artificial intelligence: A survey of expert opinion. In Fundamental issues of artificial intelligence (pp. 553–570). Springer.

  66. Nair, S., Rao, N., Mishra, S., & Patil, A. (2017). India’s charging infrastructure — Biggest single point impediment in EV adaptation in India. In 2017 IEEE transportation electrification conference (ITEC-India) (pp. 1–6). https://doi.org/10.1109/ITEC-India.2017.8333884

  67. Oke, A. (2007). Innovation types and innovation management practices in service companies. International Journal of Operations & Production Management, 27(6), 564–587.

    Google Scholar 

  68. Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up?: Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on empirical methods in natural language processing-Volume 10 (pp. 79–86).

  69. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: Machine learning in {P}ython. Journal of Machine Learning Research, 12, 2825–2830.

    Google Scholar 

  70. Peffers, K., Rothenberger, M., Tuunanen, T., & Vaezi, R. (2012). Design science research evaluation. Design Science Research in Information Systems. Advances in Theory and Practice, 398–410. https://doi.org/10.1007/978-3-642-29863-9_29 .

  71. Perrin, A. (2015). Social Media Usage: 2005–2015: 65% of Adults Now Use Social Networking Sites–a Nearly Tenfold Jump in the Past Decade. Pew Research Center, (October), 2005–2015.

  72. Platt, J. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, 10(3), 61–74.

    Google Scholar 

  73. Porter, M. F. (2001). Snowball: A language for stemming algorithms.

  74. Powers, D. M. (2011). Evaluation: From precision, recall and F-measure to ROC, Informedness, Markedness and correlation. Journal of Machine Learning Technologies, 2(1), 37–63.

    Google Scholar 

  75. Saldaña, J. (2015). The coding manual for qualitative researchers. The coding manual for qualitative researchers. https://doi.org/10.1017/CBO9781107415324.004

  76. Scheffler, T., Gontrum, J., Wegel, M., & Wendler, S. (2015). Mapping German tweets to geographic regions. Working Paper - University of Potsdam.

  77. Scheurenbrand, J., Engel, C., Peters, F., & Kühl, N. (2015). Holistically defining E-mobility: A modern approach to systematic literature reviews. Karlsruhe Service Summit, 17–27. https://doi.org/10.5445/KSP/1000045634 .

  78. Schlagwein, D., Fischbach, K., & Schoder, D. (2011). Social information systems: Review, framework, and research agenda. International Conference on Information Systems. https://doi.org/10.12980/APJTB.4.2014C1020 .

  79. Sheldon, K. M., Elliot, A. J., Kim, Y., & Kasser, T. (2001). What is satisfying about satisfying events? Testing 10 candidate psychological needs. Journal of Personality and Social Psychology, 80(2), 325–339.

    Google Scholar 

  80. Silva, C., & Ribeiro, B. (2003). The importance of stop word removal on recall values in text categorization. In Neural networks, 2003. Proceedings of the international joint conference on (Vol. 3, pp. 1661–1666).

    Google Scholar 

  81. Singhal, A. (2001). Modern information retrieval: A brief overview. IEEE Data Eng. Bull., 24(4), 35–43.

    Google Scholar 

  82. Srivastava, A. N., & Sahami, M. (2009). Text mining: Classification, clustering, and applications. CRC Press.

  83. Srivastava, R. K., Shervani, T. A., & Fahey, L. (1999). Marketing, business processes, and shareholder value: An organizationally embedded view of marketing activities and the discipline of marketing. The Journal of Marketing, 168–179.

  84. St Louis, C., & Zorlu, G. (2012). Can twitter predict disease outbreaks. BMJ, 344, e2353.

    Google Scholar 

  85. Steinwart, I., & Christmann, A. (2008). Support vector machines. Springer Science & Business Media.

  86. Stieglitz, S., Dang-Xuan, L., Bruns, A., & Neuberger, C. (2014). Social media analytics. Business & Information Systems Engineering, 6(2), 89–96.

    Google Scholar 

  87. Subramaniam, L. V., Faruquie, T. A., Ikbal, S., Godbole, S., & Mohania, M. K. (2009). Business intelligence from voice of customer. In IEEE international conference on data engineering (pp. 1391–1402).

  88. Timoshenko, A., & Hauser, J. R. (2018). Identifying customer needs from user-generated content.

  89. Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010). Predicting elections with twitter: What 140 characters reveal about political sentiment. Proceedings of the fourth international AAAI conference on weblogs and social media, 178–185. https://doi.org/10.1074/jbc.M501708200 , 280

  90. Turney, P. D. (2002). Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 417–424).

  91. Twitter (2016). About Twitter. Retrieved August 2, 2016, from https://about.twitter.com/de/company/press/milestones, last accessed 2016-08-02.

  92. Twitter (2018). Twitter Streaming API. Retrieved March 22, 2018, from https://developer.twitter.com/en/docs/tweets/filter-realtime/guides/connecting. Accessed 2019-02-01.

  93. Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics, 7(1), 91.

    Google Scholar 

  94. Webb, G. I., Pazzani, M. J., & Billsus, D. (2001). Machine learning for user modeling. User Modeling and User-Adapted Interaction, 11(1), 19–29.

    Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Niklas Kühl.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Social Information Systems Minitrack - HICCS 51

Responsible Editors: Rainer Schmidt, Rainer Alt and Selmin Nurcan

Appendix

Appendix

Fig. 5
figure5

Example of a JSON response from the deployed REST API

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kühl, N., Mühlthaler, M. & Goutier, M. Supporting customer-oriented marketing with artificial intelligence: automatically quantifying customer needs from social media. Electron Markets 30, 351–367 (2020). https://doi.org/10.1007/s12525-019-00351-0

Download citation

Keywords

  • Customer needs
  • Supervised machine learning
  • Twitter
  • Web services
  • E-mobility
  • Social information Systems
  • Marketing

JEL classification

  • C38
  • C81
  • L94
  • O32