Advertisement

KnowBots: Discovering Relevant Patterns in Chatbot Dialogues

  • Adriano RivolliEmail author
  • Catarina Amaral
  • Luís Guardão
  • Cláudio Rebelo de Sá
  • Carlos Soares
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11828)

Abstract

Chatbots have been used in business contexts as a new way of communicating with customers. They use natural language to interact with the customers, whether while offering products and services, or in the support of a specific task. In this context, an important and challenging task is to assess the effectiveness of the machine-to-human interaction, according to business’ goals. Although several analytic tools have been proposed to analyze the user interactions with chatbot systems, to the best of our knowledge they do not consider user-defined criteria, focusing on metrics of engagement and retention of the system as a whole. For this reason, we propose the KnowBots tool, which can be used to discover relevant patterns in the dialogues of chatbots, by considering specific business goals. Given the non-trivial structure of dialogues and the possibly large number of conversational records, we combined sequential pattern mining and subgroup discovery techniques to identify patterns of usage. Moreover, a friendly user-interface was developed to present the results and to allow their detailed analysis. Thus, it may serve as an alternative decision support tool for business or any entity that makes use of this type of interactions with their clients.

Keywords

Chatbot analytics Chatbot analysis Logs analysis Sequence mining Subgroup discovery 

References

  1. 1.
    Shah, K.B., Shetty, M.S., Shah, D.P., Pamnani, R.: Approaches towards building a banking assistant. Int. J. Comput. Appl. 166(11), 1–6 (2017).  https://doi.org/10.5120/ijca2017914140CrossRefGoogle Scholar
  2. 2.
    Chai, J.Y., et al.: The role of a natural language conversational interface in online sales: a case study. Int. J. Speech Technol. 4(3–4), 285–295 (2001).  https://doi.org/10.1023/A:1011316909641CrossRefzbMATHGoogle Scholar
  3. 3.
    Chakrabarti, C., Luger, G.F.: Artificial conversations for customer service chatter bots: architecture, algorithms, and evaluation metrics. Expert Syst. Appl. 42(20), 6878–6897 (2015).  https://doi.org/10.1016/j.eswa.2015.04.067CrossRefGoogle Scholar
  4. 4.
    Duivesteijn, W., Feelders, A., Knobbe, A.J.: Exceptional model mining - supervised descriptive local pattern mining with complex target concepts. Data Min. Knowl. Discov. 30(1), 47–98 (2016).  https://doi.org/10.1007/s10618-015-0403-4MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Fournier-Viger, P., Chun, J., Lin, W., Kiran, R.U., Koh, Y.S., Thomas, R.: A survey of sequential pattern mining. Data Sci. Pattern Recogn. 1, 54–77 (2017)Google Scholar
  6. 6.
    Fournier-Viger, P., Gomariz, A., Campos, M., Thomas, R.: Fast vertical mining of sequential patterns using co-occurrence information. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014. LNCS (LNAI), vol. 8443, pp. 40–52. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-06608-0_4CrossRefGoogle Scholar
  7. 7.
    Fournier-Viger, P., Gomariz, A., Gueniche, T., Soltani, A., Wu, C.W., Tseng, V.S.: SPMF: a java open-source pattern mining library. J. Mach. Learn. Res. 15(1), 3389–3393 (2014)zbMATHGoogle Scholar
  8. 8.
    Herrera, F., Carmona, C.J., González, P., del Jesús, M.J.: An overview on subgroup discovery: foundations and applications. Knowl. Inf. Syst. 29(3), 495–525 (2011).  https://doi.org/10.1007/s10115-010-0356-2CrossRefGoogle Scholar
  9. 9.
    Jia, J.: CSIEC: a computer assisted English learning chatbot based on textual knowledge and reasoning. Knowl.-Based Syst. 22(4), 249–255 (2009).  https://doi.org/10.1016/j.knosys.2008.09.001CrossRefGoogle Scholar
  10. 10.
    Jusoh, S., Al-Fawareh, H.M.: Natural language interface for online sales systems. In: 2007 International Conference on Intelligent and Advanced Systems, pp. 224–228. IEEE (2007).  https://doi.org/10.1109/ICIAS.2007.4658379
  11. 11.
    Kerly, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl.-Based Syst. 20(2), 177–185 (2007).  https://doi.org/10.1016/j.knosys.2006.11.014CrossRefGoogle Scholar
  12. 12.
    Mooney, C., Roddick, J.F.: Sequential pattern mining - approaches and algorithms. ACM Comput. Surv. 45(2), 19:1–19:39 (2013).  https://doi.org/10.1145/2431211.2431218CrossRefzbMATHGoogle Scholar
  13. 13.
    Pereira, J., Díaz, Ó.: Chatbot dimensions that matter: lessons from the trenches. In: Mikkonen, T., Klamma, R., Hernández, J. (eds.) ICWE 2018. LNCS, vol. 10845, pp. 129–135. Springer, Cham (2018).  https://doi.org/10.1007/978-3-319-91662-0_9CrossRefGoogle Scholar
  14. 14.
    Shah, H., Warwick, K., Vallverdú, J., Wu, D.: Can machines talk? comparison of eliza with modern dialogue systems. Comput. Hum. Behav. 58, 278–295 (2016).  https://doi.org/10.1016/j.chb.2016.01.004CrossRefGoogle Scholar
  15. 15.
    Shawar, B.A., Atwell, E.: Chatbots: are they really useful? LDV Forum 22(1), 29–49 (2007)Google Scholar
  16. 16.
    Souza, M., Miyagawa, T., Melo, P., Maciel, F.: Wellness programs: wearable technologies supporting healthy habits and corporate costs reduction. In: Stephanidis, C. (ed.) HCI 2017. CCIS, vol. 714, pp. 293–300. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-58753-0_44CrossRefGoogle Scholar
  17. 17.
    Toxtli, C., Monroy-Hernández, A., Cranshaw, J.: Understanding chatbot-mediated task management. In: Mandryk, R.L., Hancock, M., Perry, M., Cox, A.L. (eds.) Proceedings of the Conference on Human Factors in Computing Systems, p. 58. ACM (2018).  https://doi.org/10.1145/3173574.3173632

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Adriano Rivolli
    • 1
    • 2
    Email author
  • Catarina Amaral
    • 2
    • 4
  • Luís Guardão
    • 2
  • Cláudio Rebelo de Sá
    • 3
  • Carlos Soares
    • 4
  1. 1.ICMC-USP/UTFPRCornélio ProcópioBrazil
  2. 2.INESC TECPortoPortugal
  3. 3.University of TwenteEnschedeThe Netherlands
  4. 4.Faculty of EngineeringUniversity of PortoPortoPortugal

Personalised recommendations