Abstract
The explosive growth of information on the Internet has created a great demand for new and powerful tools to acquire useful information. The first step to retrieve information form Chinese article is word segmentation. But there are two major segmentation problems that might affect the accuracy of word segmentation performance, ambiguity and long words. In this paper, we propose a novel character-based approach, namely, dynamic N-gram (DNG) to deal with the two above problems of word segmentation and apply it to Chinese news articles to evaluate the accuracy of N-gram. The evaluation result indicated most of the readers agreed that dynamic N-gram approach could extract meaningful keywords. Even in different news categories, the keywords extraction results still have no significant difference. The primary contribution of this approach is that dynamic N-gram helps us to extract the most meaningful keywords in different types of Chinese articles without considering the number of grams.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Zhang, M., Lu, Z., Zou, C.: A Chinese Word Segmentation Based on Language Situation in Processing Ambiguous Words. Information Sciences 162, 275–285 (2004)
Fu, G., Kit, C., Webster, J.J.: Chinese Word Segmentation as Morpheme-based Lexical Chunking. Information Sciences 178, 2282–2296 (2008)
Tsai, R.T.: Chinese Text Segmentation: A Hybrid Approach Using Transductive Learning and Statistical Association Measures. Expert Systems with Applications 37(5), 3553–3560 (2010)
Haizhou, L., Baosheng, Y.: Chinese Word Segmentation. Language, Information and Computation, 212–217 (1998)
Sun, X., Zhang, Y., Matsuzaki, T., Tsuruoka, Y., Tsujii, J.: Probabilistic Chinese Word Segmentation with Non-local Information and Stochastic Training. Information Processing and Management 49(3), 626–636 (2013)
Foo, S., Li, H.: Chinese Word Segmentation and Its Effect on Information Retrieval. Information Processing & Management 40, 161–190 (2004)
Tong, X., Zhai, C., Milic-Frayling, N., Evans, D.A.: Experiments on Chinese Text Indexing-—CLARIT TREC-5 Chinese Track Report. In: TREC (1996)
Kwok, K.L.: Lexicon Effects on Chinese Information Retrieval. In: Proc. of 2nd Conf. on Empirical Methods in NLP, pp. 141–148. ACL (1997)
Sproat, R., Shih, C.: A Statistical Method for Finding Word Boundaries in Chinese Text. Computer Processing of Chinese and Oriental Languages 4(4), 336–351 (1990)
Wu, Z., Tseng, G.: Chinese Text Segmentation for Text Retrieval: Achievements and Problems. Journal of the American Society for Information Science 44(9), 532–542 (1993)
Sproat, R., Shih, C., Gale, W., Chang, N.: A Stochastic Finite-state Word-segmentation Algorithm for Chinese. Computational Linguistics 22(3), 377–404 (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Lin, RX., Yang, HL. (2014). Apply the Dynamic N-gram to Extract the Keywords of Chinese News. In: Ali, M., Pan, JS., Chen, SM., Horng, MF. (eds) Modern Advances in Applied Intelligence. IEA/AIE 2014. Lecture Notes in Computer Science(), vol 8482. Springer, Cham. https://doi.org/10.1007/978-3-319-07467-2_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-07467-2_42
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-07466-5
Online ISBN: 978-3-319-07467-2
eBook Packages: Computer ScienceComputer Science (R0)