Abstract
Due to the complexity of natural language, chatbots are prone to misinterpreting user requests. Such misinterpretations may lead the chatbot to provide answers that are not adequate responses to user request – so called false positives – potentially leading to conversational breakdown. A promising repair strategy in such cases is for the chatbot to express uncertainty and suggest likely alternatives in cases where prediction confidence falls below threshold. However, little is known about how such repair affects chatbot dialogues. We present findings from a study where a solution for expressing uncertainty and suggesting likely alternatives was implemented in a live chatbot for customer service. Chatbot dialogues (N = 700) were sampled at two points in time – immediately before and after implementation – and compared by conversational quality. Preliminary analyses suggest that introducing such a solution for conversational repair may substantially reduce the proportion of false positives in chatbot dialogues. At the same time, expressing uncertainty and suggesting likely alternatives does not seem to strongly affect the dialogue process and the likelihood of reaching a successful outcome. Based on the findings, we discuss theoretical and practical implications and suggest directions for future research.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
All dialogue examples in the paper are paraphrased.
References
Albert, S., De Ruiter, J.P.: Repair: the interface between interaction and cognition. Top. Cogn. Sci. 10(2), 279–313 (2018)
Ashktorab, Z., Jain, M., Liao, Q.V., Weisz, J.D.: Resilient chatbots: repair strategy preferences for conversational breakdowns. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, paper no. 254. ACM, New York (2019)
Bazzanella, C., Damiano, R.: The interactional handling of misunderstanding in everyday conversations. J. Pragmat. 31, 817–836 (1999)
Dingemanse, M., Blythe, J., Dirksmeyer, T.: Formats for other-initiation of repair across languages: an exercise in pragmatic typology. Stud. Lang. 38 (2014). https://doi.org/10.1075/sl.38.1.01din
Drift: The 2018 State of Chatbots Report. Technical report, Drift (2018). https://www.drift.com/blog/chatbots-report/
Ezzy, D.: Qualitative Analysis: Practice and Innovation. Routledge, London (2002)
Følstad, A., Skjuve, M., Brandtzaeg, P.B.: Different chatbots for different purposes: towards a typology of chatbots to understand interaction design. In: Bodrunova, S., et al. (eds.) INSCI 2018. LNCS, vol. 11551, pp. 145–156. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-17705-8_13
Følstad, A., Skjuve, M.: Chatbots for customer service: user experience and motivation. In: Proceedings of the 1st International Conference on Conversational User Interfaces, paper no. 1. ACM, New York (2019)
Forrester: Human vs. machines: how to stop your virtual agent from lagging behind. Technical report, Forrester (2017). https://www.amdocs.com/blog/place-digital-talks-intelligent-minds/aia-humans-vs-machines-how-to-stop-your-chatbot-from-lagging-behind
Grice, H.P.: Logic and conversation. In: Syntax and Semantics 3: Speech Acts, pp. 41–58. Academic Press, New York (1975)
Hall, E.: Conversational Design. A Book Apart, New York (2018)
Kendrick, K.H.: Other-initiated repair in English. Open Linguist. 1, 164–190 (2014)
Kocielnik, R., Amershi, S., Bennett, P.N.: Will you accept an imperfect AI?: Exploring designs for adjusting end-user expectations of AI systems. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, paper no. 411. ACM, New York (2019)
Moore, Robert J.: A natural conversation framework for conversational UX design. In: Moore, R., Szymanski, M., Arar, R., Ren, G.-J. (eds.) Studies in Conversational UX Design. HIS, pp. 181–204. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95579-7_9
Nordheim, C.B., Følstad, A., Bjørkli, C.A.: An initial model of trust in chatbots for customer service—Findings from a questionnaire study. Interact. Comput. (2019). https://doi.org/10.1093/iwc/iwz022
Sacks, H., Schegloff, E.A., Jefferson, G.: A simplest systematics for the organization of turntaking for conversation. Language 50(4), 696–735 (1974)
Schegloff, E.A.: Some sources of misunderstanding in talk-in-interaction. Linguistics 25, 201–218 (1987)
Schegloff, E.A.: Sequence Organization in Interaction: A Primer in Conversation Analysis, vol. 1. Cambridge University Press, Cambridge (2007)
Searle, J.R.: A classification of illocutionary acts. Lang. Soc. 5(1), 1–23 (1976)
Shadish, W.R., Cook, T.D., Campbell, D.T.: Experimental and quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin, Boston (2002)
Shevat, A.: Designing Bots: Creating Conversational Experiences. O’Reilly Media, Newton (2017)
Acknowledgements
This study was conducted in collaboration with SpareBank 1 SR-Bank. The work was supported by the Research Council of Norway, Grant No. 282244.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Følstad, A., Taylor, C. (2020). Conversational Repair in Chatbots for Customer Service: The Effect of Expressing Uncertainty and Suggesting Alternatives. In: Følstad, A., et al. Chatbot Research and Design. CONVERSATIONS 2019. Lecture Notes in Computer Science(), vol 11970. Springer, Cham. https://doi.org/10.1007/978-3-030-39540-7_14
Download citation
DOI: https://doi.org/10.1007/978-3-030-39540-7_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-39539-1
Online ISBN: 978-3-030-39540-7
eBook Packages: Computer ScienceComputer Science (R0)