Advertisement

Intent Detection System Based on Word Embeddings

  • Kaspars Balodis
  • Daiga Deksne
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11089)

Abstract

Intent detection is one of the main tasks of a dialogue system. In this paper we present our intent detection system that is based on FastText word embeddings and neural network classifier. We find a significant improvement in the FastText sentence vectorization. The results show that our intent detection system provides state-of-the-art results on three English datasets outperforming many popular services.

Keywords

Intent detection Dialog system Word embeddings 

Notes

Acknowledgments

The research has been supported by the European Regional Development Fund within the project “Neural Network Modelling for Inflected Natural Languages” No. 1.1.1.1/16/A/215.

References

  1. 1.
    Balodis, K.: On stopword identification in dialog systems, submitted for publicationGoogle Scholar
  2. 2.
    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Trans. Assoc. Comput. Linguist. 5, 135–146 (2017)Google Scholar
  3. 3.
    Braun, D., Hernandez-Mendez, A., Matthes, F., Langen, M.: Evaluating natural language understanding services for conversational question answering systems. In: Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp. 174–185 (2017)Google Scholar
  4. 4.
    Damnati, G., Auguste, J., Nasr, A., Charlet, D., Heinecke, J., Béchet, F.: Handling normalization issues for part-of-speech tagging of online conversational text. In: Calzolari (Conference Chair), N., (eds.) Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Paris, France, May 2018Google Scholar
  5. 5.
    Deksne, D., Skadina, I.: Error-annotated corpus of Latvian. In: Baltic HLT, pp. 163–166 (2014)Google Scholar
  6. 6.
    Fonte, F., Carlos, J., Rial, B., Nistal, M.L.: TQ-Bot: an AIML-based tutor and evaluator bot. J. Univ. Comput. Sci. 15(7), 1486–1495 (2009)Google Scholar
  7. 7.
    van der Goot, R., van Noord, G.: MoNoise: modeling noise using a modular normalization system. Comput. Linguist. Neth. J. 7, 129–144 (2017)Google Scholar
  8. 8.
    Grave, E., Bojanowski, P., Gupta, P., Joulin, A., Mikolov, T.: Learning word vectors for 157 languages. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) (2018)Google Scholar
  9. 9.
    Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 (2016)
  10. 10.
    Kalchbrenner, N., Blunsom, P.: Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584 (2013)
  11. 11.
    Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751 (2014)Google Scholar
  12. 12.
    Liu, B., Lane, I.: Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:1609.01454 (2016)
  13. 13.
    Liu, C., Xu, P., Sarikaya, R.: Deep contextual language understanding in spoken dialogue systems. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)Google Scholar
  14. 14.
    Lowe, R., Pow, N., Serban, I.V., Pineau, J.: The Ubuntu dialogue corpus: a large dataset for research in unstructured multi-turn dialogue systems. In: 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue, p. 285 (2015)Google Scholar
  15. 15.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
  16. 16.
    Náplava, J., Straka, M., Straňák, P., Hajič, J.: Diacritics restoration using neural networks. In: Calzolari (Conference Chair), N., (eds.) Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Paris, France, May 2018Google Scholar
  17. 17.
    Neves, A.M., Barros, F.A., Hodges, C.: iAIML: a mechanism to treat intentionality in AIML chatterbots. In: 18th IEEE International Conference on Tools with Artificial Intelligence 2006, ICTAI 2006, pp. 225–231. IEEE (2006)Google Scholar
  18. 18.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)Google Scholar
  19. 19.
    Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50. ELRA, Valletta, Malta, May 2010. http://is.muni.cz/publication/884893/en
  20. 20.
    Sales, J.E., Souza, L., Barzegar, S., Davis, B., Freitas, A., Handschuh, S.: Indra: A word embedding and semantic relatedness server. In: Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan, May 2018Google Scholar
  21. 21.
    Serban, I.V., Sordoni, A., Bengio, Y., Courville, A.C., Pineau, J.: Building end-to-end dialogue systems using generative hierarchical neural network models. In: AAAI, vol. 16, pp. 3776–3784 (2016)Google Scholar
  22. 22.
    Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364 (2015)
  23. 23.
    Shawar, B.A., Atwell, E.: Machine learning from dialogue corpora to generate chatbots. Expert Update J. 6(3), 25–29 (2003)Google Scholar
  24. 24.
    Vinyals, O., Le, Q.: A neural conversational model. arXiv preprint arXiv:1506.05869 (2015)
  25. 25.
    Wallace, R.: The Elements of AIML Style. Alice AI Foundation (2003)Google Scholar
  26. 26.
    Wen, T., et al.: A network-based end-to-end trainable task-oriented dialogue system. In: 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017-Proceedings of Conference, vol. 1, pp. 438–449 (2017)Google Scholar
  27. 27.
    Wen, T.H., Gasic, M., Mrkšić, N., Su, P.H., Vandyke, D., Young, S.: Semantically conditioned LSTM-based natural language generation for spoken dialogue systems. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1711–1721. Association for Computational Linguistics (2015).  https://doi.org/10.18653/v1/D15-1199, http://www.aclweb.org/anthology/D15-1199
  28. 28.
    Xu, P., Sarikaya, R.: Convolutional neural network based triangular CRF for joint intent detection and slot filling. In: IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) 2013, pp. 78–83. IEEE (2013)Google Scholar
  29. 29.
    Yang, X., Chen, Y.N., Hakkani-Tür, D., Crook, P., Li, X., Gao, J., Deng, L.: End-to-end joint learning of natural language understanding and dialogue manager. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2017, pp. 5690–5694. IEEE (2017)Google Scholar
  30. 30.
    Yao, K., Peng, B., Zhang, Y., Yu, D., Zweig, G., Shi, Y.: Spoken language understanding using long short-term memory neural networks. In: Spoken Language Technology Workshop (SLT), 2014 IEEE, pp. 189–194. IEEE (2014)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.TildeRigaLatvia
  2. 2.Faculty of ComputingUniversity of LatviaRigaLatvia

Personalised recommendations