Abstract
The increasing interest in cross-lingual and multilingual information retrieval has posed a great challenge of designing accurate information retrieval systems for Asian languages such as Chinese, Thai and Japanese. Word segmentation is one of the most important pre-processes of Chinese information processing. This paper reviews some popular word segmentation algorithms. Based on an improved Converse Chinese dictionary and an optimized reverse maximum matching algorithm, a Chinese word segmentation system is proposed. Experiments are carried out to demonstrate the substantially ameliorated accuracy and speed of the system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Kwok, K.L.: Employing multiple representations for Chinese information retrieval. Journal of the American Society for Information 50(8), 709–723 (1999)
Kwok, K.L.: Improving English and Chinese ad-hoc retrieval: A tipster text phase 3 project report. Information Retrieval 3(4), 313–338 (2000)
Spark-Jones, K.: The role of artificial intelligence in information retrieval. Journal of the American Society for Information Science 42(8), 558–565 (1991)
Fan, C.K., Tsai, W.H.: Automatic word identification in Chinese sentence by the relaxation technique. Computer Processing of Chinese and Oriental Language 1, 33–56 (1988)
Jin, G.: Critical tokenization and its properties. Computational Linguistics 4, 569–596 (1997)
Chen, A., He, J., Xu, L., Gey, F.C., Meggs, J.: Chinese text retrieval without using a dictionary. In: Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 42–49. ACM, New York (1997)
Lee, Hong, K., Lu, Q.: Text segmentation for Chinese spells checking. Journal of the American Society for Information Science 50(9), 751–759 (1999)
Huang, X., Robertson, S.: A probability approach to Chinese information retrieval: Theory and experiments. In: Proceedings of the BCS-IRSG 2000, the 22nd Annual Colloquium on Information Retrieval Research, Cambridge, England (2000)
Sun, Y.-H., He, P.-L., Nie, S., Wu, G.-Y.: A Chinese Segmentation System Based on Document Self-Machine for Identifying the Unknown Words. In: Proceedings of the Second International Conference on Machine Learning and Cybernetics, Xi’an, November 2-5, pp. 2080–2084 (2003)
Dahan, D., Brent, M.: On the Discovery of novel word-like units from utterances: An artificial language study with implications for native-language acquisition. Journal of Experimental Psychology 128, 165–185 (1999)
Cheng, K.S., Wong, K.F.: A study on word-based and integral-bit Chinese text compression algorithms. Journal of the American Society for Information Science 50(3), 218–228 (1999)
Jin, H., Wong, K.-F.: a Chinese Dictionary Construction Algorithm for Information Retrieval. ACM Transactions on Asian Language Information Processing 1(4), 281–296 (2002)
Kaiying, L.: Automatic word segmentation and tagging of Chinese texts. The Commercial Press, Beijing (2000)
Jerome, P.: The Morphology of Chinese: A Linguistic and Cognitive Approach. Cambridge University Press, Cambridge (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, L., Li, Y., Meng, J. (2006). Design of Chinese Word Segmentation System Based on Improved Chinese Converse Dictionary and Reverse Maximum Matching Algorithm. In: Feng, L., Wang, G., Zeng, C., Huang, R. (eds) Web Information Systems – WISE 2006 Workshops. WISE 2006. Lecture Notes in Computer Science, vol 4256. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11906070_17
Download citation
DOI: https://doi.org/10.1007/11906070_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-47663-4
Online ISBN: 978-3-540-47664-1
eBook Packages: Computer ScienceComputer Science (R0)