Distributional Models and Lexical Semantics in Convolution Kernels

  • Danilo Croce
  • Simone Filice
  • Roberto Basili
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7181)

Abstract

The representation of word meaning in texts is a central problem in Computational Linguistics. Geometrical models represent lexical semantic information in terms of the basic co-occurrences that words establish each other in large-scale text collections. As recent works already address, the definition of methods able to express the meaning of phrases or sentences as operations on lexical representations is a complex problem, and a still largely open issue. In this paper, a perspective centered on Convolution Kernels is discussed and the formulation of a Partial Tree Kernel that integrates syntactic information and lexical generalization is studied. The interaction of such information and the role of different geometrical models is investigated on the question classification task where the state-of-the-art result is achieved.

Keywords

Singular Value Decomposition Parse Tree Convolution Kernel Lexical Information Lexical Semantic 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Harris, Z.: Distributional structure. In: Katz, J.J., Fodor, J.A. (eds.) The Philosophy of Linguistics. Oxford University Press (1964)Google Scholar
  2. 2.
    Sahlgren, M.: The Word-Space Model. PhD thesis, Stockholm University (2006)Google Scholar
  3. 3.
    Turney, P.D., Pantel, P.: From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research 37, 141–188 (2010)MathSciNetMATHGoogle Scholar
  4. 4.
    Schutze, H.: Automatic word sense discrimination. Journal of Computational Linguistics 24, 97–123 (1998)Google Scholar
  5. 5.
    Lin, D.: Automatic retrieval and clustering of similar word. In: Proceedings of COLING-ACL, Montreal, Canada (1998)Google Scholar
  6. 6.
    Giuliano, C.: Fine-grained classification of named entities exploiting latent semantic kernels. In: Proceedings of CoNLL 2009, Stroudsburg, PA, USA, pp. 201–209 (2009)Google Scholar
  7. 7.
    Croce, D., Giannone, C., Annesi, P., Basili, R.: Towards open-domain semantic role labeling. In: ACL, pp. 237–246 (2010)Google Scholar
  8. 8.
    Pado, S., Lapata, M.: Dependency-based construction of semantic space models. Computational Linguistics 33(2) (2007)Google Scholar
  9. 9.
    Mitchell, J., Lapata, M.: Composition in distributional models of semantics. Cognitive Science 34, 1388–1429 (2010)CrossRefGoogle Scholar
  10. 10.
    Baroni, M., Lenci, A.: One distributional memory, many semantic spaces. In: Proceedings of the GEMS 2009 Workshop, GEMS 2009, Stroudsburg, PA, USA, pp. 1–8 (2009)Google Scholar
  11. 11.
    Clark, S., Pulman, S.: Combining Symbolic and Distributional Models of Meaning. In: Proceedings of the AAAI Spring Symposium on Quantum Interaction, pp. 52–55 (2007)Google Scholar
  12. 12.
    Grefenstette, E., Sadrzadeh, M.: Experimental support for a categorical compositional distributional model of meaning. In: Proceedings of EMNLP 2011, Edinburgh, Scotland, UK. (2011)Google Scholar
  13. 13.
    Haussler, D.: Convolution kernels on discrete structures. Technical report, University of Santa Cruz (1999)Google Scholar
  14. 14.
    Collins, M., Duffy, N.: New Ranking Algorithms for Parsing and Tagging: Kernels over Discrete Structures, and the Voted Perceptron. In: Proceedings of ACL 2002 (2002)Google Scholar
  15. 15.
    Bloehdorn, S., Moschitti, A.: Combined Syntactic and Semantic Kernels for Text Classification. In: Amati, G., Carpineto, C., Romano, G. (eds.) ECIR 2007. LNCS, vol. 4425, pp. 307–318. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  16. 16.
    Croce, D., Moschitti, A., Basili, R.: Structured Lexical Similarity via Convolution Kernels on Dependency Trees. In: Proceedings of EMNLP 2011 (2011)Google Scholar
  17. 17.
    Pedersen, T., Patwardhan, S., Michelizzi, J.: WordNet:Similarity - Measuring the Relatedness of Concept. In: Proc. of 5th NAACL, Boston, MA (2004)Google Scholar
  18. 18.
    Salton, G., Wong, A., Yang, C.: A vector space model for automatic indexing. Communications of the ACM 18 (1975)Google Scholar
  19. 19.
    Landauer, T., Dumais, S.: A solution to plato’s problem: The latent semantic analysis theory of acquisition, induction and representation of knowledge. Psychological Review 104 (1997)Google Scholar
  20. 20.
    Collins, M., Duffy, N.: Convolution kernels for natural language. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 625–632 (2001)Google Scholar
  21. 21.
    Moschitti, A.: Efficient Convolution Kernels for Dependency and Constituent Syntactic Trees. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 318–329. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Golub, G., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. Journal of the Society for Industrial and Applied Mathematics: Series B, Numerical AnalysisGoogle Scholar
  23. 23.
    Cristianini, N., Shawe-Taylor, J., Lodhi, H.: Latent semantic kernels. In: Brodley, C., Danyluk, A. (eds.) Proceedings of ICML 2001, 18th International Conference on Machine Learning, pp. 66–73. Williams College, Morgan Kaufmann Publishers, San Francisco, US (2001)Google Scholar
  24. 24.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press (2004)Google Scholar
  25. 25.
    Li, X., Roth, D.: Learning question classifiers. In: Proceedings of ACL 2002 (2002)Google Scholar
  26. 26.
    Johansson, R., Nugues, P.: Dependency-based syntactic–semantic analysis with PropBank and NomBank. In: Proceedings of CoNLL 2008, pp. 183–187 (2008)Google Scholar
  27. 27.
    Baroni, M., Bernardini, S., Ferraresi, A., Zanchetta, E.: The wacky wide web: a collection of very large linguistically processed web-crawled corpora. LRE 43(3), 209–226 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Danilo Croce
    • 1
  • Simone Filice
    • 1
  • Roberto Basili
    • 1
  1. 1.Department of Enterprise EngineeringUniversity of Roma, Tor VergataRomaItaly

Personalised recommendations