Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees

  • Alessandro Moschitti
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4212)


In this paper, we provide a study on the use of tree kernels to encode syntactic parsing information in natural language learning. In particular, we propose a new convolution kernel, namely the Partial Tree (PT) kernel, to fully exploit dependency trees. We also propose an efficient algorithm for its computation which is futhermore sped-up by applying the selection of tree nodes with non-null kernel. The experiments with Support Vector Machines on the task of semantic role labeling and question classification show that (a) the kernel running time is linear on the average case and (b) the PT kernel improves on the other tree kernels when applied to the appropriate parsing paradigm.


Support Vector Machine Node Pair Parse Tree Convolution Kernel Semantic Role 


  1. 1.
    Jackendoff, R.: Semantic Structures, Current Studies in Linguistics series. The MIT Press, Cambridge (1990)Google Scholar
  2. 2.
    Collins, M., Duffy, N.: New ranking algorithms for parsing and tagging: Kernels over discrete structures, and the voted perceptron. In: ACL (2002)Google Scholar
  3. 3.
    Zelenko, D., Aone, C., Richardella, A.: Kernel methods for relation extraction. JMLR (2003)Google Scholar
  4. 4.
    Cumby, C., Roth, D.: Kernel methods for relational learning. In: ICML (2003)Google Scholar
  5. 5.
    Culotta, A., Sorensen, J.: Dependency tree kernels for relation extraction. In: Proceedings ACL, Barcelona, Spain (2004)Google Scholar
  6. 6.
    Moschitti, A.: A study on convolution kernels for shallow semantic parsing. In: Proceedings of ACL, Barcelona, Spain (2004)Google Scholar
  7. 7.
    Vishwanathan, S., Smola, A.: Fast kernels on strings and trees. In: Proceedings of NIPS (2002)Google Scholar
  8. 8.
    Kingsbury, P., Palmer, M.: From Treebank to PropBank. In: Proceedings of LREC, Las Palmas, Spain (2002)Google Scholar
  9. 9.
    Fillmore, C.J.: Frame semantics. In: Linguistics in the Morning Calm. (1982)Google Scholar
  10. 10.
    Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of english: The Penn Treebank. CLJ (1993)Google Scholar
  11. 11.
    Collins, M.: Three generative, lexicalized models for statistical parsing. In: Proceedings of the ACL, Somerset, New Jersey (1997)Google Scholar
  12. 12.
    Klein, D., Manning, C.D.: Fast exact inference with a factored model for natural language parsing. In: NIPS (2002)Google Scholar
  13. 13.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  14. 14.
    Haussler, D.: Convolution kernels on discrete structures. Technical report ucs-crl-99-10, University of California, Santa Cruz (1999)Google Scholar
  15. 15.
    Gildea, D., Jurasfky, D.: Automatic labeling of semantic roles. CLJ (2002)Google Scholar
  16. 16.
    Zhang, D., Lee, W.S.: Question classification using support vector machines. In: Proceedings of SIGIR (2003)Google Scholar
  17. 17.
    Li, X., Roth, D.: Learning question classifiers: The role of semantic information. JNLE (2005)Google Scholar
  18. 18.
    Kazama, J., Torisawa, K.: Speeding up training with tree kernels for node relation labeling. In: Proceedings of EMNLP, Toronto, Canada (2005)Google Scholar
  19. 19.
    Kudo, T., Suzuki, J., Isozaki, H.: Boosting-based parse reranking with subtree features. In: Proceedings ACL 2005 (2005)Google Scholar
  20. 20.
    Carreras, X., Màrquez, L.: Introduction to the CoNLL-2005 shared task: Semantic role labeling. In: Proceedings of CoNLL-2005 (2005)Google Scholar
  21. 21.
    Joachims, T.: Making large-scale SVM learning practical. In: Schölkopf, B., Burges, C., Smola, A. (eds.) Advances in Kernel Methods - Support Vector Learning (1999)Google Scholar
  22. 22.
    Pradhan, S., Hacioglu, K., Krugler, V., Ward, W., Martin, J.H., Jurafsky, D.: Support vector learning for semantic argument classification. MLJ (2005)Google Scholar
  23. 23.
    Kudo, T., Matsumoto, Y.: Fast methods for kernel-based text analysis. In: Proceedings of ACL (2003)Google Scholar
  24. 24.
    Suzuki, J., Isozaki, H., Maeda, E.: Convolution kernels with feature selection for natural language processing tasks. In: Proceedings of ACL, Spain (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Alessandro Moschitti
    • 1
  1. 1.Department of Computer ScienceUniversity of Rome “Tor Vergata”Italy

Personalised recommendations