Abstract
This paper provides an overview of the 2019 Language and Intelligence Challenge (LIC 2019), which assesses the ability of machines to understand language and use language to interact with humans. The challenge comprised three tasks: information extraction (IE), knowledge-driven dialogue, and machine reading comprehension (MRC), all providing large-scale Chinese datasets and open-source baseline systems. There were 2,376 teams that took part in the challenge, with a total of 6,212 system runs submitted. The participating systems performed quite well, offering a 21.65% increase over the baseline in IE, a 37.40% increase in the dialogue task, and a 34.09% increase in MRC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
The workers were collected from a Chinese crowdsourcing platform http://test.baidu.com/. The workers were paid 2.5 Chinese Yuan per conversation.
- 8.
- 9.
- 10.
References
Ghazvininejad, M., Brockett, C., Chang, M.W., Dolan, B., Gao, J., Yih, W.t., Galley, M.: A knowledge-grounded neural conversation model. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
He, W., et al.: DuReader: a Chinese machine reading comprehension dataset from real-world applications. In: Proceedings of the Workshop on Machine Reading for Question Answering, pp. 37–46 (2018)
Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. Text Summarization Branches Out (2004)
Liu, K., Liu, L., Liu, J., Lyu, Y., She, Q., Zhang, Q., Shi, Y.: Overview of 2018 NLP challenge on machine reading comprehension. J. Chin. Inf. Process. 32(10), 118–129 (2018). (in Chinese)
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 311–318 (2002)
Seo, M., Kembhavi, A., Farhadi, A., Hajishirzi, H.: Bidirectional attention flow for machine comprehension. In: International Conference on Learning Representations (2017)
Wu, W., et al.: Proactive human-machine conversation with explicit conversation goals. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (2019)
Yang, A., Liu, K., Liu, J., Yajuan, L., Li, S.: Adaptations of ROUGE and BLEU to better evaluate machine reading comprehension task. In: Proceedings of the Workshop on Machine Reading for Question Answering, pp. 98–104 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Q. et al. (2019). An Overview of the 2019 Language and Intelligence Challenge. In: Tang, J., Kan, MY., Zhao, D., Li, S., Zan, H. (eds) Natural Language Processing and Chinese Computing. NLPCC 2019. Lecture Notes in Computer Science(), vol 11839. Springer, Cham. https://doi.org/10.1007/978-3-030-32236-6_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-32236-6_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32235-9
Online ISBN: 978-3-030-32236-6
eBook Packages: Computer ScienceComputer Science (R0)
