Efficient and Robust Phrase Chunking Using Support Vector Machines

  • Yu-Chieh Wu
  • Jie-Chi Yang
  • Yue-Shi Lee
  • Show-Jane Yen
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4182)


Automatic text chunking is a task which aims to recognize phrase structures in natural language text. It is the key technology of knowledge-based system where phrase structures provide important syntactic information for knowledge representation. Support Vector Machine (SVM-based) phrase chunking system had been shown to achieve high performance for text chunking. But its inefficiency limits the actual use on large dataset that only handles several thousands tokens per second. In this paper, we firstly show that the state-of-the-art performance (94.25) in the CoNLL-2000 shared task based on conventional SVM learning. However, the off-the-shelf SVM classifiers are inefficient when the number of phrase types scales to high. Therefore, we present two novel methods that make the system substantially faster in terms of training and testing while only results in a slightly decrease of system performance. Experimental result shows that our method achieves 94.09 in F rate, which handles 13000 tokens per second in the CoNLL-2000 chunking task.


Support Vector Machine Noun Phrase Verb Phrase Representation Style Consistent Matrix 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ando, R.K., Zhang, T.: A high-performance semi-supervised learning method for text chunking. In: Proceedings of 43rd Annual Meetings of the Association for Computational Linguistics, pp. 1–9 (2005)Google Scholar
  2. 2.
    Brill, E.: Transformation-based error-driven learning and natural language processing: a case study in part of speech tagging. Computational Linguistics 21(4), 543–565 (1995)Google Scholar
  3. 3.
    Carreras, X., Marquez, L.: Phrase recognition by filtering and ranking with perceptrons. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP) (2003)Google Scholar
  4. 4.
    Carreras, X., Marquez, L., Castro, J.: Filtering-ranking perceptron learning for partial parsing. Machine Learning Journal 59, 1–31 (2005)Google Scholar
  5. 5.
    Giménez, J., Márquez, L.: Fast and accurate Part-of-Speech tagging: the SVM approach revisited. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP), pp. 158–165 (2003)Google Scholar
  6. 6.
    Joachims, T.: Text categorization with support vector machines: learning with many relevant features. In: Proceedings of the European Conference on Machine Learning, pp. 137–142 (1998)Google Scholar
  7. 7.
    Kudoh, T., Matsumoto, Y.: Chunking with support vector machines. In: The Proceedings of the 2nd Meetings of the North American Chapter and the Association for the Computational Linguistics, pp. 192–199 (2001)Google Scholar
  8. 8.
    Molina, A., Pla, F.: Shallow Parsing using Specialized HMMs. Journal of Machine Learning Research, 595–613 (2002)Google Scholar
  9. 9.
    Platt, J.C., Cristianini, N., Shawe-Taylor, J.: Large margin dags for multiclass classification. Advanced in Neural Information Processing Systems 12, 547–553 (2000)Google Scholar
  10. 10.
    Ramshaw, L.A., Marcus, M.P.: Text chunking using transformation based learning. In: Proceedings of the 3rd Workshop on Very Large Corpora, pp. 82–94 (1995)Google Scholar
  11. 11.
    Sagae, K., Lavie, A., MacWhinney, B.: Automatic Measurement of Syntactic Development in Child Language. In: Proceedings of 43rd Annual Meetings of the Association for Computational Linguistics, pp. 197–204 (2005)Google Scholar
  12. 12.
    Tjong Kim Sang, E.F., Buchholz, S.: Introduction to the CoNLL 2000 shared task: chunking. In: Proceedings of Conference on Natural Language Learning (CoNLL), pp. 127–132 (2000)Google Scholar
  13. 13.
    Tjong Kim Sang, E.F.: Memory-based shallow parsing. Journal of Machine Learning Research, 559–594 (2002)Google Scholar
  14. 14.
    Watanabe, T., Sumita, E., Okuno, H.G.: Chunk-based statistical translation. In: Proceedings of 41st Annual Meetings of the Association for Computational Linguistics, pp. 303–310 (2003)Google Scholar
  15. 15.
    Wu, Y.-C., Chang, C.-H., Lee, Y.-S.: A general and multi-lingual phrase chunking model based on masking method. In: Proceedings of 7th International Conference on Intelligent Text Processing and Computational Linguistics, pp. 144–155 (2006)Google Scholar
  16. 16.
    Zhang, T., Damerau, F., Johnson, D.: Text Chunking based on a Generalization Winnow. Journal of Machine Learning Research 2, 615–637 (2002)MATHCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Yu-Chieh Wu
    • 1
  • Jie-Chi Yang
    • 2
  • Yue-Shi Lee
    • 3
  • Show-Jane Yen
    • 3
  1. 1.Department of Computer Science and Information EngineeringNational Central University 
  2. 2.Graduate Institute of Network Learning TechnologyNational Central UniversityJhongli City, Taoyuan CountyTaiwan, R.O.C.
  3. 3.Department of Computer Science and Information EngineeringMing Chuan UniversityTaoyuanTaiwan, R.O.C.

Personalised recommendations