Skip to main content
SpringerLink
Log in

Designing from experience: on the adoption of chatbot conversational design practices by practitioners

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

One of the essential aspects of chatbot management is user retention, which is a consequence of good service provision. In order to reach this goal, there needs to be a well-thought-out conversational design. This process involves everything from understanding the domain to designing and conceiving how the conversations between machine and user can develop, aiming for greater acceptance of the chatbot through the most spontaneous and natural dialog possible. Related literature on general design revealed that personal experiences affect software design decisions. This work aims to evaluate how participants’ built-up perceptions and experiences with chatbots and their previous experiences assuming specific positions in a software development team influence their adoption of conversational guidelines proposed by a guide. Invited practitioners were asked to develop conversations without and with the conversational guidelines, and then they were interviewed about this process. We ran objective, narrative, and thematic analyses on interview transcripts and participants’ conversations. Most participants had negative experiences with chatbots, which greatly influenced how they viewed some guidelines. Furthermore, their experiences in specific software development positions also influenced their design and adoption of guidelines. Results revealed that personal experience is a significant moderating variable in conversational design. This work’s main contribution is providing knowledge on how practitioners enter the world of chatbot design and what concerns should exist in preparing introductory material considering their personal experiences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Data Availability

The data that supports the findings of this work is available on Zenodo at https://doi.org/10.5281/zenodo.7523384 [34].

Notes

  1. https://sonix.ai/

References

  1. Abdellatif A, Costa D, Badran K et al (2020) Challenges in chatbot development: a study of stack overflow posts. In: Proceedings of the 17th international conference on mining software repositories, pp 174–185

  2. Aghajani E, Nagy C, Linares-Vásquez M, et al (2020) Software documentation: the practitioners’ perspective. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE), IEEE, pp 590–601

  3. Androutsopoulou A, Karacapilidis N, Loukis E et al (2019) Transforming the communication between citizens and government through AI-guided chatbots. Gov Inf Q 36(2):358–367

    Article  Google Scholar 

  4. Baym N (2015) Personal connections in the digital age. Digital Media and Society, Polity Press, https://books.google.com.br/books?id=4_1RCgAAQBAJ

  5. Brachten F, Kissmer T, Stieglitz S (2021) The acceptance of chatbots in an enterprise context - a survey study. Int J Inf Manage 60(102):375

  6. Canedo ED, Bonifácio R, Okimoto MV, et al (2020) Work practices and perceptions from women core developers in OSS communities. In: Baldassarre MT, Lanubile F, Kalinowski M et al (eds) ESEM ’20: ACM / IEEE International Symposium on Empirical Software Engineering and Measurement, Bari, Italy, October 5–7, 2020. ACM, pp 26:1–26:11. https://doi.org/10.1145/3382494.3410682

  7. Canedo ED, Mendes FF, Cerqueira AJ, et al (2021) Breaking one barrier at a time: how women developers cope in a men-dominated industry. In: Vasconcellos CD, Roggia KG, Collere V et al (eds) 35th Brazilian symposium on software engineering, SBES 2021, Joinville, Santa Catarina, Brazil, 27 September 2021 - 1 October 2021. ACM, pp 378–387. https://doi.org/10.1145/3474624.3474638

  8. Clandinin DJ, Connelly FM (2004) Narrative inquiry: experience and story in qualitative research. John Wiley & Sons

  9. Clemmensen T, Hertzum M, Yang J et al (2013) Do usability professionals think about user experience in the same way as users and developers do? In: IFIP conference on human-computer interaction, Springer, pp 461–478

  10. Cook TD, Campbell DT, Shadish W (2002) Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Boston, MA

    Google Scholar 

  11. Desjardins A, Ball A (2018) Revealing tensions in autobiographical design in hci. In: Proceedings of the 2018 designing interactive systems conference, pp 753–764

  12. Dwivedi YK, Hughes L, Baabdullah AM et al (2022) Metaverse beyond the hype: multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. Int J Inf Manage 66:102–542

  13. Følstad A, Brandtzæg PB (2017) Chatbots and the new world of HCI. Interactions 24(4):38–42. https://doi.org/10.1145/3085558

    Article  Google Scholar 

  14. Følstad A, Skjuve M (2019) Chatbots for customer service: user experience and motivation. In: Proceedings of the 1st international conference on conversational user interfaces, pp 1–9

  15. Følstad A, Nordheim CB, Bjørkli CA (2018) What makes users trust a chatbot for customer service? An exploratory interview study. In: Bodrunova SS (ed) Internet Science - 5th International Conference, INSCI 2018, Lecture Notes in Computer Science, vol 11193. Springer, St. Petersburg, Russia, pp 194–208. https://doi.org/10.1007/978-3-030-01437-7_16

  16. Google (2021) What is conversation design? https://developers.google.com/assistant/conversation-design/what-is-conversation-design

  17. Henman P (2020) Improving public services using artificial intelligence: possibilities, pitfalls, governance. Asia Pacific Journal of Public Administration 42(4):209–221

    Article  Google Scholar 

  18. Hill J, Randolph Ford W, Farreras IG (2015) Real conversations with artificial intelligence: a comparison between human-human online conversations and human-chatbot conversations. Comput Hum Behav 49:245–250

  19. Iivari N, Kuutti K (2017) Critical design research and information technology: searching for empowering design. In: Proceedings of the 2017 conference on designing interactive systems, pp 983–993

  20. Kiger ME, Varpio L (2020) Thematic analysis of qualitative data: AMEE guide no. 131. Medical Teacher 42(8):846–854

  21. Laumer S, Gubler FT, Racheva AA et al (2019) Use cases for conversational agents: an interview-based study. In: AMCIS

  22. Leiva G, Maudet N, Mackay W et al (2019) Enact: reducing designer-developer breakdowns when prototyping custom interactions. ACM Transactions on Computer-Human Interaction (TOCHI) 26(3):1–48

    Article  Google Scholar 

  23. Liebrecht C, Sander L, Hooijdonk Cv (2020) Too informal? how a chatbot’s communication style affects brand attitude and quality of interaction. In: International workshop on chatbot research and design, Springer, pp 16–31

  24. Lin YT, Hertzum M (2020) How do designers make user-experience design decisions? In: International conference on human-computer interaction, Springer, pp 188–198

  25. Lommatzsch A (2018) A next generation chatbot-framework for the public administration. In: International conference on innovations for community services, Springer, pp 127–141

  26. MarketsandMarkets (2021) Conversational AI market size, share and global market forecast to 2026. https://www.marketsandmarkets.com/Market-Reports/conversational-ai-market-49043506.html?gclid=Cj0KCQjw0PWRBhDKARIsAPKHFGh6B5mjeTgniXB0CG6HkceFBDGMDxe0O9HbeqJEWG0scGsKD6UQd1MaAquZEALw_wcB

  27. Matthews T, Judge T, Whittaker S (2012) How do designers and user experience professionals actually perceive and use personas? In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 1219–1228

  28. Meyer-Waarden L, Pavone G, Poocharoentou T et al (2020) How service quality influences customer acceptance and usage of chatbots? SMR-Journal of Service Management Research 4(1):35–51

    Google Scholar 

  29. Nasheeda A, Abdullah HB, Krauss SE et al (2019) Transforming transcripts into stories: a multimethod approach to narrative analysis. Int J Qual Methods 18(1609406919856):797. https://doi.org/10.1177/1609406919856797

  30. Neustaedter C, Sengers P (2012) Autobiographical design in HCI research: designing and learning through use-it-yourself. In: Proceedings of the designing interactive systems conference, pp 514–523

  31. Ollerenshaw JA, Creswell JW (2002) Narrative research: a comparison of two restorying data analysis approaches. Qual Inq 8(3):329–347. https://doi.org/10.1177/10778004008003008

    Article  Google Scholar 

  32. Petriv Y, Erlenheim R, Tsap V, et al (2019) Designing effective chatbot solutions for the public sector: a case study from Ukraine. In: International conference on electronic governance and open society: challenges in Eurasia, Springer, pp 320–335

  33. Silva GRS, Canedo ED (2022) Towards user-centric guidelines for chatbot conversational design. International Journal of Human-Computer Interaction 1–23. https://doi.org/10.1080/10447318.2022.2118244

  34. Silva GRS, Canedo ED (2023) Supplementary material for “designing from experience: on the adoption of chatbot conversational design practices by practitioners”. https://doi.org/10.5281/zenodo.7523384

  35. Svikhnushina E, Placinta A, Pu P (2021) User expectations of conversational chatbots based on online reviews. Designing Interactive Systems Conference 2021:1481–1491

    Google Scholar 

  36. Zamora J (2017) I’m sorry, dave, i’m afraid I can’t do that: chatbot perception and expectations. In: Wrede B, Nagai Y, Komatsu T et al (eds) Proceedings of the 5th international conference on Human Agent Interaction, HAI 2017. ACM, Bielefeld, Germany, pp 253–260. https://doi.org/10.1145/3125739.3125766

  37. Zhang X, Wakkary R (2014) Understanding the role of designers’ personal experiences in interaction design practice. In: Proceedings of the 2014 conference on designing interactive systems, pp 895–904

Download references

Acknowledgements

The authors would like to express sincere thanks to all volunteers participating in the experiment.

Funding

This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Brasilia (UnB), Brasília, DF, Brazil

    Geovana Ramos Sousa Silva & Edna Dias Canedo

Authors
  1. Geovana Ramos Sousa Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edna Dias Canedo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geovana Ramos Sousa Silva.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Silva, G.R.S., Canedo, E.D. Designing from experience: on the adoption of chatbot conversational design practices by practitioners. Pers Ubiquit Comput (2024). https://doi.org/10.1007/s00779-024-01800-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00779-024-01800-0

Keywords

Search

Navigation