Abstract
In this paper, we give an overview for the shared task at the 5th CCF Conference on Natural Language Processing & Chinese Computing (NLPCC 2016): Chinese word segmentation for micro-blog texts. Different with the popular used newswire datasets, the dataset of this shared task consists of the relatively informal micro-texts. Besides, we also use a new psychometric-inspired evaluation metric for Chinese word segmentation, which addresses to balance the very skewed word distribution at different levels of difficulty. The data and evaluation codes can be downloaded from https://github.com/FudanNLP/NLPCC-WordSeg-Weibo.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Feng, H., Chen, K., Deng, X., Zheng, W.: Accessor variety criteria for Chinese word extraction. Comput. Linguist. 30(1), 75–93 (2004)
Leng, Y., Liu, W., Wang, S., Wang, X.: A feature-rich CRF segmenter for chinese micro-blog. In: Proceedings of the Fifth Conference on Natural Language Processing and Chinese Computing and the Twenty Fourth International Conference on Computer Processing of Oriental Languages (2016)
Qian, P., Qiu, X., Huang, X.: A new psychometric-inspired evaluation metric for Chinese word segmentation. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (2016). http://aclweb.org/anthology/P/P16/P16-1206.pdf
Qiu, X., Qian, P., Yin, L., Wu, S., Huang, X.: Overview of the NLPCC 2015 shared task: Chinese word segmentation and POS tagging for micro-blog texts. In: Li, J., Ji, H., Zhao, D., Feng, Y. (eds.) NLPCC 2015. LNCS (LNAI), vol. 9362, pp. 541–549. Springer, Heidelberg (2015). doi:10.1007/978-3-319-25207-0_50
Qiu, X., Zhang, Q., Huang, X.: FudanNLP: a toolkit for Chinese natural language processing. In: Proceedings of Annual Meeting of the Association for Computational Linguistics (2013)
Xia, Q., Li, Z., Chao, J., Zhang, M.: Word segmentation on micro-blog texts with external lexicon and heterogeneous data. In: Proceedings of the Fifth Conference on Natural Language Processing and Chinese Computing & the Twenty Fourth International Conference on Computer Processing of Oriental Languages (2016)
Zhou, Q., Ma, L., Zheng, Z., Wang, Y., Wang, X.: Recurrent neural word segmentation with tag inference. In: Proceedings of the Fifth Conference on Natural Language Processing and Chinese Computing & the Twenty Fourth International Conference on Computer Processing of Oriental Languages (2016)
Acknowledgement
We are very grateful to the students from our lab for their efforts to annotate and check the data. We would also like to thank the participants for their valuable feedbacks and comments. This work was partially funded by National Natural Science Foundation of China (No. 61532011 and 61672162), the National High Technology Research and Development Program of China (No. 2015AA015408).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Qiu, X., Qian, P., Shi, Z. (2016). Overview of the NLPCC-ICCPOL 2016 Shared Task: Chinese Word Segmentation for Micro-Blog Texts. In: Lin, CY., Xue, N., Zhao, D., Huang, X., Feng, Y. (eds) Natural Language Understanding and Intelligent Applications. ICCPOL NLPCC 2016 2016. Lecture Notes in Computer Science(), vol 10102. Springer, Cham. https://doi.org/10.1007/978-3-319-50496-4_84
Download citation
DOI: https://doi.org/10.1007/978-3-319-50496-4_84
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50495-7
Online ISBN: 978-3-319-50496-4
eBook Packages: Computer ScienceComputer Science (R0)