Advertisement

Towards Open Domain Chatbots—A GRU Architecture for Data Driven Conversations

  • Åsmund Kamphaug
  • Ole-Christoffer Granmo
  • Morten Goodwin
  • Vladimir I. Zadorozhny
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10750)

Abstract

Understanding of textual content, such as topic and intent recognition, is a critical part of chatbots, allowing the chatbot to provide relevant responses. Although successful in several narrow domains, the potential diversity of content in broader and more open domains renders traditional pattern recognition techniques inaccurate. In this paper, we propose a novel deep learning architecture for content recognition that consists of multiple levels of gated recurrent units (GRUs). The architecture is designed to capture complex sentence structure at multiple levels of abstraction, seeking content recognition for very wide domains, through a distributed scalable representation of content. To evaluate our architecture, we have compiled 10 years of questions and answers from a youth information service, \(200\ 083\) questions spanning a wide range of content, altogether 289 topics, involving law, health, and social issues. Despite the relatively open domain data set, our architecture is able to accurately categorize the 289 intents and topics. Indeed, it provides roughly an order of magnitude higher accuracy compared to more classical content recognition techniques, such as SVM, Naive Bayes, random forest, and K-nearest neighbor, which all seem to fail on this challenging open domain dataset.

References

  1. 1.
  2. 2.
  3. 3.
  4. 4.
    Microsoft Bot Framework. https://docs.botframework.com/en-us/
  5. 5.
    Abdul-Kader, S.A., Woods, J.: Survey on chatbot design techniques in speech conversation systems. Int. J. Adv. Comput. Sci. Appl. (IJACSA) 6, 72–80 (2015)Google Scholar
  6. 6.
    Bayser, M.G.d., Cavalin, P., Souza, R., Braz, A., Candello, H., Pinhanez, C., Briot, J.P.: A hybrid architecture for multi-party conversational systems. arXiv:1705.01214 [cs.CL] (2017)
  7. 7.
    Breiman, L.: Random forest. Mach. Learn. 45, 5–32 (1999)CrossRefzbMATHGoogle Scholar
  8. 8.
    Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
  9. 9.
    Ferrucci, D.A.: Introduction to this is Watson. IBM J. Res. Dev. 56(3.4), 1:1–1:15 (2012)CrossRefGoogle Scholar
  10. 10.
    Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space Odyssey. IEEE Trans. Neural Netw. Learn. Syst. 28, 2222–2232 (2017)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Lee, D., Oh, K.J., Choi, H.J.: The chatbot feels you - a counseling service using emotional response generation. In: 2017 IEEE International Conference on Big Data and Smart Computing, BigComp 2017 (2017)Google Scholar
  12. 12.
    Marietto, M.d.G.B., Aguiar, R.V., Barbosa, G.d.O., Botelho, W.T., Pimentel, E., Franca, R.d.S., Silva, V.L.d.: Artificial intelligence markup language a brief tutorial. Int. J. Comput. Sci. Eng. Surv. 4(3), 1–20 (2013)Google Scholar
  13. 13.
    Rokach, L., Maimom, O.: Data Mining with Decision Trees: Theory and Applications. World Scientific, Singapore (2014)CrossRefGoogle Scholar
  14. 14.
    Shawar, B.A., Atwell, E.: ALICE chatbot: trials and outputs. Computacion y Sistemas 19(4), 625–632 (2015)Google Scholar
  15. 15.
    Kojouharov, S.: Ultimate Guide to Leveraging NLP & Machine Learning for your Chatbot. https://chatbotslife.com/ultimate-guide-to-leveraging-nlp-machine-learning-for-you-chatbot-531ff2dd870c
  16. 16.
    Walker, J.: Chatbot Comparison Facebook, Microsoft, Amazon, and Google (2017). https://www.techemergence.com/chatbot-comparison-facebook-microsoft-amazon-google/
  17. 17.
    Weizenbaum, J.: Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman & Co., New York (1976)Google Scholar
  18. 18.
    Xu, A., Liu, Z., Guo, Y., Sinha, V., Akkiraju, R.: A new chatbot for customer service on social media. In: Proceedings of the CHI Conference on Human Factors in Computing Systems (2017)Google Scholar
  19. 19.
    Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for document classification. In: HLT-NAACL, pp. 1480–1489 (2016)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Åsmund Kamphaug
    • 1
  • Ole-Christoffer Granmo
    • 1
  • Morten Goodwin
    • 1
  • Vladimir I. Zadorozhny
    • 1
    • 2
  1. 1.Centre for Artificial Intelligence ResearchUniversity of AgderKristiansandNorway
  2. 2.School of Computing and InformationUniversity of PittsburghPittsburghUSA

Personalised recommendations