KnowBots: Discovering Relevant Patterns in Chatbot Dialogues

  • Adriano RivolliEmail author
  • Catarina Amaral
  • Luís Guardão
  • Cláudio Rebelo de Sá
  • Carlos Soares
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11828)


Chatbots have been used in business contexts as a new way of communicating with customers. They use natural language to interact with the customers, whether while offering products and services, or in the support of a specific task. In this context, an important and challenging task is to assess the effectiveness of the machine-to-human interaction, according to business’ goals. Although several analytic tools have been proposed to analyze the user interactions with chatbot systems, to the best of our knowledge they do not consider user-defined criteria, focusing on metrics of engagement and retention of the system as a whole. For this reason, we propose the KnowBots tool, which can be used to discover relevant patterns in the dialogues of chatbots, by considering specific business goals. Given the non-trivial structure of dialogues and the possibly large number of conversational records, we combined sequential pattern mining and subgroup discovery techniques to identify patterns of usage. Moreover, a friendly user-interface was developed to present the results and to allow their detailed analysis. Thus, it may serve as an alternative decision support tool for business or any entity that makes use of this type of interactions with their clients.


Chatbot analytics Chatbot analysis Logs analysis Sequence mining Subgroup discovery 


  1. 1.
    Shah, K.B., Shetty, M.S., Shah, D.P., Pamnani, R.: Approaches towards building a banking assistant. Int. J. Comput. Appl. 166(11), 1–6 (2017). Scholar
  2. 2.
    Chai, J.Y., et al.: The role of a natural language conversational interface in online sales: a case study. Int. J. Speech Technol. 4(3–4), 285–295 (2001). Scholar
  3. 3.
    Chakrabarti, C., Luger, G.F.: Artificial conversations for customer service chatter bots: architecture, algorithms, and evaluation metrics. Expert Syst. Appl. 42(20), 6878–6897 (2015). Scholar
  4. 4.
    Duivesteijn, W., Feelders, A., Knobbe, A.J.: Exceptional model mining - supervised descriptive local pattern mining with complex target concepts. Data Min. Knowl. Discov. 30(1), 47–98 (2016). Scholar
  5. 5.
    Fournier-Viger, P., Chun, J., Lin, W., Kiran, R.U., Koh, Y.S., Thomas, R.: A survey of sequential pattern mining. Data Sci. Pattern Recogn. 1, 54–77 (2017)Google Scholar
  6. 6.
    Fournier-Viger, P., Gomariz, A., Campos, M., Thomas, R.: Fast vertical mining of sequential patterns using co-occurrence information. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014. LNCS (LNAI), vol. 8443, pp. 40–52. Springer, Cham (2014). Scholar
  7. 7.
    Fournier-Viger, P., Gomariz, A., Gueniche, T., Soltani, A., Wu, C.W., Tseng, V.S.: SPMF: a java open-source pattern mining library. J. Mach. Learn. Res. 15(1), 3389–3393 (2014)zbMATHGoogle Scholar
  8. 8.
    Herrera, F., Carmona, C.J., González, P., del Jesús, M.J.: An overview on subgroup discovery: foundations and applications. Knowl. Inf. Syst. 29(3), 495–525 (2011). Scholar
  9. 9.
    Jia, J.: CSIEC: a computer assisted English learning chatbot based on textual knowledge and reasoning. Knowl.-Based Syst. 22(4), 249–255 (2009). Scholar
  10. 10.
    Jusoh, S., Al-Fawareh, H.M.: Natural language interface for online sales systems. In: 2007 International Conference on Intelligent and Advanced Systems, pp. 224–228. IEEE (2007).
  11. 11.
    Kerly, A., Hall, P., Bull, S.: Bringing chatbots into education: towards natural language negotiation of open learner models. Knowl.-Based Syst. 20(2), 177–185 (2007). Scholar
  12. 12.
    Mooney, C., Roddick, J.F.: Sequential pattern mining - approaches and algorithms. ACM Comput. Surv. 45(2), 19:1–19:39 (2013). Scholar
  13. 13.
    Pereira, J., Díaz, Ó.: Chatbot dimensions that matter: lessons from the trenches. In: Mikkonen, T., Klamma, R., Hernández, J. (eds.) ICWE 2018. LNCS, vol. 10845, pp. 129–135. Springer, Cham (2018). Scholar
  14. 14.
    Shah, H., Warwick, K., Vallverdú, J., Wu, D.: Can machines talk? comparison of eliza with modern dialogue systems. Comput. Hum. Behav. 58, 278–295 (2016). Scholar
  15. 15.
    Shawar, B.A., Atwell, E.: Chatbots: are they really useful? LDV Forum 22(1), 29–49 (2007)Google Scholar
  16. 16.
    Souza, M., Miyagawa, T., Melo, P., Maciel, F.: Wellness programs: wearable technologies supporting healthy habits and corporate costs reduction. In: Stephanidis, C. (ed.) HCI 2017. CCIS, vol. 714, pp. 293–300. Springer, Cham (2017). Scholar
  17. 17.
    Toxtli, C., Monroy-Hernández, A., Cranshaw, J.: Understanding chatbot-mediated task management. In: Mandryk, R.L., Hancock, M., Perry, M., Cox, A.L. (eds.) Proceedings of the Conference on Human Factors in Computing Systems, p. 58. ACM (2018).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Adriano Rivolli
    • 1
    • 2
    Email author
  • Catarina Amaral
    • 2
    • 4
  • Luís Guardão
    • 2
  • Cláudio Rebelo de Sá
    • 3
  • Carlos Soares
    • 4
  1. 1.ICMC-USP/UTFPRCornélio ProcópioBrazil
  2. 2.INESC TECPortoPortugal
  3. 3.University of TwenteEnschedeThe Netherlands
  4. 4.Faculty of EngineeringUniversity of PortoPortoPortugal

Personalised recommendations