Skip to main content
Book cover

International Conference on Design Science Research in Information Systems and Technology

DESRIST 2020: Designing for Digital Transformation. Co-Creating Services with Citizens and Industry pp 126–141Cite as

Exploring Design Principles for Enterprise Chatbots: An Analytic Hierarchy Process Study

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12388)

Abstract

Chatbots have attracted tremendous interest in recent years and are increasingly employed in form of enterprise chatbots (ECBs) (i.e., chatbots used in the explicit context of enterprise systems). Although ECBs substantially differ in their design requirements from, for example, more common and widely deployed customer service chatbots, only few studies exist that specifically investigate and provide guidance for the design of ECBs. To address this emerging gap, we accumulated existing design knowledge from previous studies and created a list of 26 design features (DFs) which we integrated into 6 design principles (DPs). Subsequently, 36 practitioners from an IT consulting company which are experienced in using ECBs evaluated the importance of the DPs and DFs following the Analytic Hierarchy Process method. Our results provide evidence that DPs and DFs promoting usability and flexibility are ranked more important than DPs and DFs promoting socialness and human likeness. These findings provide valuable insights, as they are partially contrary to some existing studies investigating the importance of social cues of chatbots in other domains. Overall, the identified lists of DPs and DFs and their importance rankings provide guidance for the design of ECBs and can serve as a basis for future research projects.

Keywords

  • Chatbot
  • Enterprise system
  • AHP
  • Design principle
  • Design feature

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-64823-7_13
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   39.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-64823-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   54.99
Price excludes VAT (USA)
Learn about institutional subscriptions

References

  1. Adam, M., Wessel, M., Benlian, A.: AI-based chatbots in customer service and their effects on user compliance. Electron. Mark. 9(2), 204 (2020). https://doi.org/10.1007/s12525-020-00414-7

  2. André, E., et al.: Humane anthropomorphic agents: the quest for the outcome measure; position paper. In: AIS SIGPrag, Munich, 15–18 December 2019, 2019 pre-ICIS Workshop Proceedings “Values and Ethics in the Digital Age”, Munich, 14 December 2019 (2019)

    Google Scholar 

  3. Ashktorab, Z., Jain, M., Liao, Q.V., Weisz, J.D.: Resilient chatbots. repair strategy preferences for conversational breakdowns. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, pp. 1–12. ACM Press, New York (2019)

    Google Scholar 

  4. Avula, S., Chadwick, G., Arguello, J., Capra, R.: SearchBots. In: Shah, C., Belkin, N.J., Byström, K., Huang, J., Scholer, F. (eds.) Proceedings of the 2018 Conference on Human Information Interaction & Retrieval - CHIIR 2018. The 2018 Conference, New Brunswick, NJ, USA, 11–15 March 2018, pp. 52–61. ACM Press, New York (2018). https://doi.org/10.1145/3176349.3176380

  5. Baskerville, R., Pries-Heje, J.: Design theory projectability. IS&O 2014. IAICT, vol. 446, pp. 219–232. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45708-5_14

    CrossRef  Google Scholar 

  6. Baskerville, R., Baiyere, A., Gregor, S., Hevner, A., Rossi, M.: Design science research contributions: finding a balance between artifact and theory. J. Assoc. Inf. Syst. 19(5), 358–376 (2018)

    Google Scholar 

  7. Benlian, A.: Is traditional, open-source, or on-demand first choice? Developing an AHP-based framework for the comparison of different software models in office suites selection. Eur. J. Inf. Syst. 20(5), 542–559 (2011). https://doi.org/10.1057/ejis.2011.14

    CrossRef  Google Scholar 

  8. Bickmore, T.W., Picard, R.W.: Establishing and maintaining long-term human-computer relationships. ACM Trans. Comput.-Hum. Interact. 12(2), 293–327 (2005)

    CrossRef  Google Scholar 

  9. Byrne, D.E.: The Attraction Paradigm. Academic Press, New York (1971)

    Google Scholar 

  10. Callejas, Z., López-Cózar, R., Abalos, N., Griol, D.: Affective conversational agents: the role of personality and emotion in spoken interactions. In: Conversational Agents and Natural Language Interaction: Techniques and Effective Practices, pp. 203–222 (2011). https://doi.org/10.4018/978-1-60960-617-6.ch009

  11. Chandra, L., Seidel, S., Gregor, S.: Prescriptive knowledge in IS research: conceptualizing design principles in terms of materiality, action, and boundary conditions. In: 48th Hawaii International Conference on System Sciences, pp. 4039–4048 (2015)

    Google Scholar 

  12. Chattaraman, V., Kwon, W.-S., Gilbert, J.E., Ross, K.: Should AI-based, conversational digital assistants employ social- or task-oriented interaction style? A task-competency and reciprocity perspective for older adults. Comput. Hum. Behav. 90, 315–330 (2019). https://doi.org/10.1016/j.chb.2018.08.048

    CrossRef  Google Scholar 

  13. Dale, R.: The return of the chatbots. Nat. Lang. Eng. 22(5), 811–817 (2016). https://doi.org/10.1017/S1351324916000243

    CrossRef  Google Scholar 

  14. Feine, J., Gnewuch, U., Morana, S., Maedche, A.: A taxonomy of social cues for conversational agents. Int. J. Hum.-Comput. Stud. 132, 138–161 (2019). https://doi.org/10.1016/j.ijhcs.2019.07.009

    CrossRef  Google Scholar 

  15. Feine, J., Morana, S., Maedche, A.: Designing a chatbot social cue configuration system. In: Proceedings of the 40th International Conference on Information Systems (ICIS), AISel, Munich (2019)

    Google Scholar 

  16. Feine, J., Morana, S., Maedche, A.: Leveraging machine-executable descriptive knowledge in design science research – the case of designing socially-adaptive chatbots. In: Tulu, B., Djamasbi, S., Leroy, G. (eds.) DESRIST 2019. LNCS, vol. 11491, pp. 76–91. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19504-5_6

    CrossRef  Google Scholar 

  17. Feine, J., Morana, S., Gnewuch, U.: Measuring service encounter satisfaction with customer service chatbots using sentiment analysis. In: 14. Internationale Tagung Wirtschaftsinformatik (WI2019) (2019)

    Google Scholar 

  18. Feine, J., Gnewuch, U., Morana, S., Maedche, A.: Gender bias in chatbot design. In: Følstad, A., et al. (eds.) CONVERSATIONS 2019. LNCS, vol. 11970, pp. 79–93. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-39540-7_6

    CrossRef  Google Scholar 

  19. Gnewuch, U., Morana, S., Maedche, A.: Towards designing cooperative and social conversational agents for customer service. In: Proceedings of the 38th International Conference on Information Systems (ICIS), AISel, Seoul (2017)

    Google Scholar 

  20. Gnewuch, U., Morana, S., Adam, M., Maedche, A.: Faster is not always better: understanding the effect of dynamic response delays in human-chatbot interaction. In: Proceedings of the 26th European Conference on Information Systems (ECIS), Portsmouth, United Kingdom, 23–28 June (2018)

    Google Scholar 

  21. Gregor, S., Hevner, A.R.: Positioning and presenting design science research for maximum impact. MIS Q. 37(2), 337–355 (2013)

    CrossRef  Google Scholar 

  22. Jain, M., Kumar, P., Kota, R., Patel, S.N.: Evaluating and informing the design of chatbots. In: Koskinen, I., Lim, Y.-K., Cerratto-Pargman, T., Chow, K., Odom, W. (eds.) Proceedings of the 2018 Conference on Designing Interactive Systems, DIS 2018, Hong Kong, 9–13 June 2018, The 2018, Hong Kong, China, 9–13 June 2018, pp. 895–906. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3196709.3196735

  23. Karlsson, J., Wohlin, C., Regnell, B.: An evaluation of methods for prioritizing software requirements. Inf. Softw. Technol. 39(14–15), 939–947 (1998). https://doi.org/10.1016/S0950-5849(97)00053-0

    CrossRef  Google Scholar 

  24. Kimani, E., Rowan, K., McDuff, D., Czerwinski, M., Mark, G.: A conversational agent in support of productivity and wellbeing at work. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 1–7 (2019)

    Google Scholar 

  25. Kocielnik, R., Avrahami, D., Marlow, J., Lu, D., Hsieh, G.: Designing for workplace reflection: a chat and voice-based conversational agent. In: Proceedings of the 2018 Designing Interactive Systems Conference (DIS 2018), pp. 881–894. ACM (2018)

    Google Scholar 

  26. Lechler, R., Stöckli, E., Rietsche, R., Uebernickel, F.: Looking beneath the tip of the iceberg: the two-sided nature of Chatbots and their roles for digital feedback exchange. In: Proceedings of the 27th European Conference on Information Systems (ECIS), Stockholm & Uppsala, Sweden (2019)

    Google Scholar 

  27. Lee, J.C., Myers, M.D.: Dominant actors, political agendas, and strategic shifts over time: a critical ethnography of an enterprise systems implementation. J. Strateg. Inf. Syst. 13(4), 355–374 (2004). https://doi.org/10.1016/j.jsis.2004.11.005

    CrossRef  Google Scholar 

  28. Liao, Q.V., Davis, M., Geyer, W., Muller, M., Shami, N.S.: What can you do? Studying social-agent orientation and agent proactive interactions with an agent for employees. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, pp. 264–275. ACM, New York (2016). https://doi.org/10.1145/2901790.2901842

  29. Liao, Q.V., et al.: All work and no play? In: Mandryk, R., Hancock, M., Perry, M., Cox, A. (eds.) Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI 2018, 21–26 April 2018, pp. 1–13. ACM Press, New York (2018). https://doi.org/10.1145/3173574.3173577

  30. Maedche, A., et al.: AI-based digital assistants. Bus. Inf. Syst. Eng. 61(4), 535–544 (2019). https://doi.org/10.1007/s12599-019-00600-8

    CrossRef  Google Scholar 

  31. Maedche, A., Gregor, S., Morana, S., Feine, J.: Conceptualization of the problem space in design science research. In: Tulu, B., Djamasbi, S., Leroy, G. (eds.) DESRIST 2019. LNCS, vol. 11491, pp. 18–31. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-19504-5_2

    CrossRef  Google Scholar 

  32. McGregor, M., Tang, J.C.: More to meetings: challenges in using speech-based technology to support meetings. In: Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, pp. 2208–2220 (2017)

    Google Scholar 

  33. McTear, M.F.: The rise of the conversational interface: a new kid on the block? In: Quesada, J.F., Martín Mateos, F.J., López-Soto, T. (eds.) FETLT 2016. LNCS (LNAI), vol. 10341, pp. 38–49. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69365-1_3

    CrossRef  Google Scholar 

  34. Meth, H., Mueller, B., Maedche, A.: Designing a requirement mining system. J. Assoc. Inf. Syst. 16(9), 799 (2015)

    Google Scholar 

  35. Meyer von Wolff, R., Hobert, S., Schumann, M.: How may I help you? - state of the art and open research questions for chatbots at the digital workplace. In: Proceedings of the 52nd Hawaii International Conference on System Sciences, pp. 95–104 (2019)

    Google Scholar 

  36. Nass, C., Moon, Y.: Machines and mindlessness. Social responses to computers. J. Soc. Issues 56(1), 81–103 (2000). https://doi.org/10.1111/0022-4537.00153

  37. Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research methodology for information systems research. J. Manag. Inf. Syst. 24(3), 45–77 (2007). https://doi.org/10.2753/MIS0742-1222240302

    CrossRef  Google Scholar 

  38. Qiu, L., Benbasat, I.: Evaluating anthropomorphic product recommendation agents. A social relationship perspective to designing information systems. J. Manag. Inf. Syst. 25(4), 145–181 (2009)

    CrossRef  Google Scholar 

  39. Rietz, T., Benke, I., Maedche, A.: The impact of anthropomorphic and functional chatbot design features in enterprise collaboration systems on user acceptance. In: 14. International Conference on Wirtschaftsinformatik (WI2019) (2019)

    Google Scholar 

  40. Rupp, C.: Requirements-Engineering und -Management, 5th edn. Hanser, Munich (2014)

    Google Scholar 

  41. Saaty, T.L.: The Analytic Hierarchy Process. McGraw-Hill, New York (1980)

    MATH  Google Scholar 

  42. Sein, M., Henfridsson, O., Purao, S., Rossi, M., Lindgren, R.: Action design research. MIS Q. 35, 37–56 (2011). https://doi.org/10.2307/23043488

    CrossRef  Google Scholar 

  43. Sjöström, J., Aghaee, N., Dahlin, M., Ågerfalk, P.: Designing chatbots for higher education practice. In: International Conference on Information Systems Education and Research, AISel, San Francisco, CA, USA (2018)

    Google Scholar 

  44. Stoeckli, E., Dremel, C., Uebernickel, F., Brenner, W.: How affordances of chatbots cross the chasm between social and traditional enterprise systems. Electron. Mark. 30(2), 369–403 (2019). https://doi.org/10.1007/s12525-019-00359-6

    CrossRef  Google Scholar 

  45. Toxtli, C., Monroy-Hernández, A., Cranshaw, J.: Understanding chatbot-mediated task management. In: Mandryk, R., Hancock, M., Perry, M., Cox, A. (eds.) Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, 21–26 April 2018, pp. 1–6. ACM Press, New York (2018)

    Google Scholar 

  46. Tremblay, M.C., Berndt, D.J.: Focus groups for artifact refinement and evaluation in design research. Commun. Assoc. Inf. Syst. 26 (2010). https://doi.org/10.17705/1cais.02627

  47. Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design science research. Eur. J. Inf. Syst. 25(1), 77–89 (2016). https://doi.org/10.1057/ejis.2014.36

    CrossRef  Google Scholar 

  48. vom Brocke, J., Maaß, W., Buxmann, P., Maedche, A., Leimeister, J.M., Pecht, G.: Future work and enterprise systems. Bus. Inf. Syst. Eng. 60(4), 357–366 (2018). https://doi.org/10.1007/s12599-018-0544-2

    CrossRef  Google Scholar 

  49. Watson, H.J.: Preparing for the cognitive generation of decision support. MIS Q. Exec. 16(3), 153–169 (2017)

    Google Scholar 

  50. Zhang, A.X., Cranshaw, J.: Making sense of group chat through collaborative tagging and summarization. Proc. ACM Hum.-Comput. Interact. 2(CSCW), 1–27 (2018)

    Google Scholar 

Download references

Acknowledgement

We want to thank Florian S. for his support in conducting the study as well as the associate editor and the anonymous reviewers for their valuable input to further improve the quality of the paper.

Author information

Authors and Affiliations

  1. Institute of Information Systems and Marketing (IISM), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

    Jasper Feine, Ivo Benke & Alexander Maedche

  2. Institute of Information Systems and Electronic Services, Technical University of Darmstadt, Darmstadt, Germany

    Martin Adam & Alexander Benlian

Authors
  1. Jasper Feine
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Adam
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ivo Benke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Maedche
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alexander Benlian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jasper Feine .

Editor information

Editors and Affiliations

  1. University of Agder, Kristiansand, Norway

    Sara Hofmann

  2. University of Paderborn, Paderborn, Germany

    Prof. Dr. Oliver Müller

  3. Aalto University, Aalto, Finland

    Prof. Dr. Matti Rossi

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Feine, J., Adam, M., Benke, I., Maedche, A., Benlian, A. (2020). Exploring Design Principles for Enterprise Chatbots: An Analytic Hierarchy Process Study. In: Hofmann, S., Müller, O., Rossi, M. (eds) Designing for Digital Transformation. Co-Creating Services with Citizens and Industry. DESRIST 2020. Lecture Notes in Computer Science(), vol 12388. Springer, Cham. https://doi.org/10.1007/978-3-030-64823-7_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64823-7_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64822-0

  • Online ISBN: 978-3-030-64823-7

  • eBook Packages: Computer ScienceComputer Science (R0)