Skip to main content
View expanded cover

International Conference on Applications of Natural Language to Information Systems

NLDB 2021: Natural Language Processing and Information Systems pp 367–375Cite as

Comparing MultiLingual and Multiple MonoLingual Models for Intent Classification and Slot Filling

  • 707 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 12801)

Abstract

With the momentum of conversational AI for enhancing client-to-business interactions, chatbots are sought in various domains, including FinTech where they can automatically handle requests for opening/closing bank accounts or issuing/terminating credit cards. Since they are expected to replace emails and phone calls, chatbots must be capable to deal with diversities of client populations. In this work, we focus on the variety of languages, in particular in multilingual countries. Specifically, we investigate the strategies for training deep learning models of chatbots with multilingual data. We perform experiments for the specific tasks of Intent Classification and Slot Filling in financial domain chatbots and assess the performance of mBERT multilingual model vs multiple monolingual models.

Keywords

  • Chatbots
  • Multilingualism
  • Intent classification
  • Slot filling

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-80599-9_32
  • Chapter length: 9 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-80599-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.

Notes

  1. 1.

    https://github.com/RasaHQ/financial-demo.

  2. 2.

    Available at https://github.com/Trustworthy-Software/BCS-dataset.

  3. 3.

    Further information on Rasa models: https://rasa.com/docs/rasa/components/.

  4. 4.

    https://github.com/google-research/bert/blob/master/multilingual.md.

  5. 5.

    https://github.com/saffsd/langid.py.

References

  1. Abbet, C., et al.: Churn intent detection in multilingual chatbot conversations and social media. arXiv preprint arXiv:1808.08432 (2018)

  2. Bocklisch, T., Faulkner, J., Pawlowski, N., Nichol, A.: Rasa: Open source language understanding and dialogue management. arXiv preprint arXiv:1712.05181 (2017)

  3. Costello, C., Lin, R., Mruthyunjaya, V., Bolla, B., Jankowski, C.: Multi-layer ensembling techniques for multilingual intent classification. arXiv preprint arXiv:1806.07914 (2018)

  4. Dahl, D.A., et al.: Expanding the scope of the atis task: the atis-3 corpus. In: HUMAN LANGUAGE TECHNOLOGY: Proceedings of a Workshop held at Plainsboro, New Jersey, 8–11 March 1994 (1994)

    Google Scholar 

  5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. In: 2019 Conference of the North American Chapter of the ACL: Human Language Technologies (2019)

    Google Scholar 

  6. Lauly, S., Larochelle, H., Khapra, M.M., Ravindran, B., Raykar, V., Saha, A., et al.: An autoencoder approach to learning bilingual word representations. arXiv preprint arXiv:1402.1454 (2014)

  7. Schuster, S., Gupta, S., Shah, R., Lewis, M.: Cross-lingual transfer learning for multilingual task oriented dialog. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 3795–3805 (2019)

    Google Scholar 

  8. Upadhyay, S., Faruqui, M., Tür, G., Dilek, H., Heck, L.: (Almost) zero-shot cross-lingual spoken language understanding. In: 2018 IEEE ICASSP, pp. 6034–6038 (2018). https://doi.org/10.1109/ICASSP.2018.8461905

  9. Upadhyay, S., Faruqui, M., Tür, G., Dilek, H.T., Heck, L.: (Almost) zero-shot cross-lingual spoken language understanding. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6034–6038. IEEE (2018)

    Google Scholar 

  10. Wan, X.: Co-training for cross-lingual sentiment classification. In: Joint Conference of the 47th Annual Meeting of the ACL, pp. 235–243 (2009)

    Google Scholar 

  11. Xu, W., Haider, B., Mansour, S.: End-to-end slot alignment and recognition for cross-lingual NLU. In: Proceedings of EMNLP 2020, November 2020, pp. 5052–5063. ACL, Online (2020). https://doi.org/10.18653/v1/2020.emnlp-main.410

  12. Zhou, G., He, T., Zhao, J.: Bridging the language gap: learning distributed semantics for cross-lingual sentiment classification. In: International Conference on Natural Language Processing and Chinese Computing, pp. 138–149. Springer, Heidelberg (2014)

    Google Scholar 

  13. Zhou, H., Chen, L., Shi, F., Huang, D.: Learning bilingual sentiment word embeddings for cross-language sentiment classification. In: 53rd Annual Meeting of the ACL and the 7th International Joint Conference on NLP, pp. 430–440 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Luxembourg, 6 rue Richard Coudenhove-Kalergi, 1359, Luxembourg, Luxembourg

    Cedric Lothritz, Kevin Allix, Bertrand Lebichot, Lisa Veiber, Tegawendé F. Bissyandé & Jacques Klein

Authors
  1. Cedric Lothritz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Allix
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bertrand Lebichot
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisa Veiber
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tegawendé F. Bissyandé
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jacques Klein
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cedric Lothritz .

Editor information

Editors and Affiliations

  1. Conservatoire National des Arts et Métiers, Paris, France

    Elisabeth Métais

  2. University of Derby, Derby, UK

    Farid Meziane

  3. German Research Center for Artificial Intelligence, Saarbrücken, Germany

    Helmut Horacek

  4. University of Hertfordshire, Hatfield, UK

    Dr. Epaminondas Kapetanios

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Lothritz, C., Allix, K., Lebichot, B., Veiber, L., Bissyandé, T.F., Klein, J. (2021). Comparing MultiLingual and Multiple MonoLingual Models for Intent Classification and Slot Filling. In: Métais, E., Meziane, F., Horacek, H., Kapetanios, E. (eds) Natural Language Processing and Information Systems. NLDB 2021. Lecture Notes in Computer Science(), vol 12801. Springer, Cham. https://doi.org/10.1007/978-3-030-80599-9_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-80599-9_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-80598-2

  • Online ISBN: 978-3-030-80599-9

  • eBook Packages: Computer ScienceComputer Science (R0)