Find out how to access preview-only content
Computational Linguistics and Intelligent Text Processing
Volume 5449 of the series Lecture Notes in Computer Science pp 170-182
Combining Language Modeling and Discriminative Classification for Word Segmentation
- Dekang LinAffiliated withGoogle, Inc.
Abstract
Generative language modeling and discriminative classification are two main techniques for Chinese word segmentation. Most previous methods have adopted one of the techniques. We present a hybrid model that combines the disambiguation power of language modeling and the ability of discriminative classifiers to deal with out-of-vocabulary words. We show that the combined model achieves 9% error reduction over the discriminative classifier alone.
Keywords
Segmentation Maximum Entropy Language Model- Title
- Combining Language Modeling and Discriminative Classification for Word Segmentation
- Book Title
- Computational Linguistics and Intelligent Text Processing
- Book Subtitle
- 10th International Conference, CICLing 2009, Mexico City, Mexico, March 1-7, 2009. Proceedings
- Pages
- pp 170-182
- Copyright
- 2009
- DOI
- 10.1007/978-3-642-00382-0_14
- Print ISBN
- 978-3-642-00381-3
- Online ISBN
- 978-3-642-00382-0
- Series Title
- Lecture Notes in Computer Science
- Series Volume
- 5449
- Series ISSN
- 0302-9743
- Publisher
- Springer Berlin Heidelberg
- Copyright Holder
- Springer-Verlag Berlin Heidelberg
- Additional Links
- Topics
- Keywords
-
- Segmentation
- Maximum Entropy
- Language Model
- Industry Sectors
- eBook Packages
- Editors
-
- Alexander Gelbukh (16)
- Editor Affiliations
-
- 16. National Polytechnic Institute, Center for Computing Research
- Authors
-
- Dekang Lin (17)
- Author Affiliations
-
- 17. Google, Inc., 1600 Amphitheater Parkway, Mountain View, CA, USA, 94043
Continue reading...
To view the rest of this content please follow the download PDF link above.