Skip to main content
SpringerLink
Log in

Leyzer: A Dataset for Multilingual Virtual Assistants

  • Conference paper
  • First Online:
Text, Speech, and Dialogue (TSD 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12284))

Included in the following conference series:

Abstract

In this article we present the Leyzer dataset, a multilingual text corpus designed to study multilingual and cross-lingual natural language understanding (NLU) models and the strategies of localization of virtual assistants. The proposed corpus consists of 20 domains across three languages: English, Spanish and Polish, with 186 intents and a wide range of samples, ranging from 1 to 672 sentences per intent. We describe the data generation process, including creation of grammars and forced parallelization. We present a detailed analysis of the created corpus. Finally, we report the results for two localization strategies: train-on-target and zero-shot learning using multilingual BERT models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Named after Ludwik Lejzer Zamenhof, a Polish linguist and the inventor of the international language Esperanto, the most widely used constructed international auxiliary language in the world. https://en.wikipedia.org/wiki/L._L._Zamenhof.

  2. 2.

    A computer-assisted translation tool: https://omegat.org/.

References

  1. Budzianowski, P., et al.: MultiWOZ - a large-scale multi-domain wizard-of-Oz dataset for task-oriented dialogue modelling. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 5016–5026. Association for Computational Linguistics (2018). https://www.aclweb.org/anthology/D18-1547

  2. Campagna, G., Ramesh, R., Xu, S., Fischer, M., Lam, M.S.: Almond: the architecture of an open, crowdsourced, privacy-preserving, programmable virtual assistant. In: Proceedings of the 26th International Conference on World Wide Web, pp. 341–350 (2017)

    Google Scholar 

  3. Cettolo, M., Corazza, A., De Mori, R.: Language portability of a speech understanding system. Comput. Speech Lang. 12(1), 1–21 (1998)

    Article  Google Scholar 

  4. Chen, Q., Zhuo, Z., Wang, W.: BERT for joint intent classification and slot filling (2019)

    Google Scholar 

  5. Coucke, A., et al.: Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv preprint arXiv:1805.10190 (2018)

  6. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 6th International Conference on Learning Representations (ICRL 2015), San Diego, CA (2015)

    Google Scholar 

  7. Kuchaiev, O., et al.: NeMo: a toolkit for building AI applications using neural modules (2019)

    Google Scholar 

  8. Larson, S., et al.: An evaluation dataset for intent classification and out-of-scope prediction. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), Hong Kong, China (2019)

    Google Scholar 

  9. Liu, X., Eshghi, A., Swietojanski, P., Rieser, V.: Benchmarking natural language understanding services for building conversational agents. arXiv preprint arXiv:1903.05566 (2019)

  10. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  11. Price, P.: Evaluation of spoken language systems: the ATIS domain. In: Proceedings of the Speech and Natural Language Workshop, Hidden Valley, PA (1990)

    Google Scholar 

  12. Schuster, S., Gupta, S., Shah, R., Lewis, M.: Cross-lingual transfer learning for multilingual task oriented dialog. In: Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-HLT 2019), Minneapolis, MN (2019)

    Google Scholar 

  13. Wolf, T., et al.: Huggingface’s transformers: State-of-the-art natural language processing. ArXiv abs/1910.03771 (2019)

    Google Scholar 

Download references

Acknowledgements

We thank Małgorzata Misiaszek for her help in verifying the quality of our corpus and improving its consistency.

Author information

Authors and Affiliations

  1. Samsung R&D Institute Poland, Warsaw, Poland

    Marcin Sowański

  2. Warsaw University of Technology, Warsaw, Poland

    Marcin Sowański & Artur Janicki

Authors
  1. Marcin Sowański
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Artur Janicki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Marcin Sowański or Artur Janicki .

Editor information

Editors and Affiliations

  1. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Petr Sojka

  2. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Ivan Kopeček

  3. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Karel Pala

  4. Faculty of Informatics, Masaryk University, Brno, Czech Republic

    Aleš Horák

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sowański, M., Janicki, A. (2020). Leyzer: A Dataset for Multilingual Virtual Assistants. In: Sojka, P., Kopeček, I., Pala, K., Horák, A. (eds) Text, Speech, and Dialogue. TSD 2020. Lecture Notes in Computer Science(), vol 12284. Springer, Cham. https://doi.org/10.1007/978-3-030-58323-1_51

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-58323-1_51

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-58322-4

  • Online ISBN: 978-3-030-58323-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation