Apply the Dynamic N-gram to Extract the Keywords of Chinese News
The explosive growth of information on the Internet has created a great demand for new and powerful tools to acquire useful information. The first step to retrieve information form Chinese article is word segmentation. But there are two major segmentation problems that might affect the accuracy of word segmentation performance, ambiguity and long words. In this paper, we propose a novel character-based approach, namely, dynamic N-gram (DNG) to deal with the two above problems of word segmentation and apply it to Chinese news articles to evaluate the accuracy of N-gram. The evaluation result indicated most of the readers agreed that dynamic N-gram approach could extract meaningful keywords. Even in different news categories, the keywords extraction results still have no significant difference. The primary contribution of this approach is that dynamic N-gram helps us to extract the most meaningful keywords in different types of Chinese articles without considering the number of grams.
KeywordsChinese Word Segmentation Dynamic N-gram Information Retrieval
Unable to display preview. Download preview PDF.
- 2.Fu, G., Kit, C., Webster, J.J.: Chinese Word Segmentation as Morpheme-based Lexical Chunking. Information Sciences 178, 2282–2296 (2008)Google Scholar
- 4.Haizhou, L., Baosheng, Y.: Chinese Word Segmentation. Language, Information and Computation, 212–217 (1998)Google Scholar
- 7.Tong, X., Zhai, C., Milic-Frayling, N., Evans, D.A.: Experiments on Chinese Text Indexing-—CLARIT TREC-5 Chinese Track Report. In: TREC (1996)Google Scholar
- 8.Kwok, K.L.: Lexicon Effects on Chinese Information Retrieval. In: Proc. of 2nd Conf. on Empirical Methods in NLP, pp. 141–148. ACL (1997)Google Scholar
- 9.Sproat, R., Shih, C.: A Statistical Method for Finding Word Boundaries in Chinese Text. Computer Processing of Chinese and Oriental Languages 4(4), 336–351 (1990)Google Scholar
- 11.Sproat, R., Shih, C., Gale, W., Chang, N.: A Stochastic Finite-state Word-segmentation Algorithm for Chinese. Computational Linguistics 22(3), 377–404 (1996)Google Scholar