Language Resources and Evaluation

, Volume 43, Issue 2, pp 105–121 | Cite as

Classification of semantic relations between nominals

  • Roxana Girju
  • Preslav Nakov
  • Vivi Nastase
  • Stan Szpakowicz
  • Peter Turney
  • Deniz Yuret


The NLP community has shown a renewed interest in deeper semantic analyses, among them automatic recognition of semantic relations in text. We present the development and evaluation of a semantic analysis task: automatic recognition of relations between pairs of nominals in a sentence. The task was part of SemEval-2007, the fourth edition of the semantic evaluation event previously known as SensEval. Apart from the observations we have made, the long-lasting effect of this task may be a framework for comparing approaches to the task. We introduce the problem of recognizing relations between nominals, and in particular the process of drafting and refining the definitions of the semantic relations. We show how we created the training and test data, list and briefly describe the 15 participating systems, discuss the results, and conclude with the lessons learned in the course of this exercise.


Semantic relations Nominals Classification SemEval 



We thank Eneko Agirre, Lluís Márquez and Richard Wicentowski, the organizers of SemEval-2007, for their guidance and prompt support in all organizational matters. Marti Hearst offered valuable advice throughout the task description and debates on definitions of semantic relations. We also thank all the participants in the SemEval-2007 Task 4 evaluation, without whom this research would not have been possible.


  1. Cafarella, M., Banko, M., & Etzioni, O. (2006). Relational web search. Technical Report 2006-04-02, University of Washington, Department of Computer Science and Engineering.Google Scholar
  2. Girju, R. (2001). Answer fusion with on-line ontology development. In Proceedings of North American chapter of the association for computational linguistics (NAACL-01)—Student Research Workshop. Pittsburgh, PA.Google Scholar
  3. Girju, R., Moldovan, D., Tatu, M., & Antohe, D. (2005). On the semantics of noun compounds. Computer Speech and Language, 19, 479–496.Google Scholar
  4. Hearst, M. (1992). Automatic acquisition of hyponyms from large text corpora. In Proceedings on 14th international conference on computational linguistics (COLING-92). pp. 539–545.Google Scholar
  5. Kim, S. N., & Baldwin, T. (2005). Automatic interpretation of noun compounds using WordNet similarity. In The international joint conference on natural language processing (IJCNLP). Jeju, Korea, pp. 945–956.Google Scholar
  6. Labov, W. (1973). The boundaries of words and their meanings. In Variation in the form and use of language: A sociolinguistic reader. pp. 29–62.Google Scholar
  7. Lapata, M. (2002). The disambiguation of nominalizations. Computational Linguistics, 28(3), 357–388.CrossRefGoogle Scholar
  8. Lapata, M., & Keller, F. (2005). Web-based models for natural language processing. ACM Transactions on Speech and Language Processing, 2, 1–31.CrossRefGoogle Scholar
  9. Lewis, D. (1991). Evaluating text categorization. In Proceedings of the speech and natural language workshop. Asilomar, pp. 312–318.Google Scholar
  10. Moldovan, D., Badulescu, A., Tatu, M., Antohe, D., & Girju, R. (2004). Models for the semantic classification of noun phrases. In Proceedings of computational lexical semantics workshop at HLT-NAACL 2004 (pp. 60–67). Boston, MA.Google Scholar
  11. Nakov, P., & Hearst, M. (2006). Using verbs to characterize noun-noun relations. In Proceedings on twelfth international conference in artificial intelligence (AIMSA-06) (pp. 233–244). Varna, Bulgaria.Google Scholar
  12. Nastase, V., Sayyad-Shirabad, J., Sokolova, M., & Szpakowicz, S. (2006). Learning noun-modifier semantic relations with corpus-based and WordNet-based features. In Proceedings on 21st national conference on artificial intelligence (AAAI 2006) (pp. 781–787). Boston, MA.Google Scholar
  13. Nastase, V., & Szpakowicz, S. (2003). Exploring noun-modifier semantic relations. In Fifth international workshop on computational semantics (IWCS-5) (pp. 285–301). Tilburg, The Netherlands.Google Scholar
  14. Pantel, P., & Pennacchiotti, M. (2006). Espresso: Leveraging generic patterns for automatically harvesting semantic relations. In The international computational linguistics conference/association for computational linguistics meeting (COLING/ACL) (pp. 113–120). Sydney, Australia.Google Scholar
  15. Pennacchiotti, M., & Pantel, P. (2006). Ontologizing semantic relations. In The international computational linguistics conference/association for computational linguistics meeting (COLING/ACL) (pp. 793–800). Sydney, Australia.Google Scholar
  16. Rosario, B., & Hearst, M. (2001). Classifying the semantic relations in noun-compounds via domain-specific lexical hierarchy. In Proceedings on 2001 conference on empirical methods in natural language processing (EMNLP-01). pp. 82–90.Google Scholar
  17. Rosario, B., Hearst, M., & Fillmore, C. (2002). The descent of hierarchy, and selection in relational semantics. In Proceedings on 40th annual meeting of the association for computational linguistics (ACL-02) (pp. 417–424). Philadelphia, PA.Google Scholar
  18. Stephens, M., Palakal, M., Mukhopadhyay, S., & Raje, R. (2001). Detecting gene relations from MEDLINE abstracts. In Proceedings on sixth annual Pacific symposium on biocomputing. pp. 483–496.Google Scholar
  19. Tatu, M., & Moldovan, D. (2005). A semantic approach to recognizing textual entailment. In Proceedings on human language technology conference and conference on empirical methods in natural language processing (HLT/EMNLP 2005) (pp. 371–378). Vancouver, Canada.Google Scholar
  20. Turney, P. (2005). Measuring semantic similarity by latent relational analysis. In Proceedings on nineteenth international joint conference on artificial intelligence (IJCAI-05) (pp. 1136–1141). Edinburgh, Scotland.Google Scholar
  21. Turney, P., & Littman, M. (2005). Corpus-based learning of analogies and semantic relations. Machine Learning, 60(1–3), 251–278.CrossRefGoogle Scholar

Copyright information

© Her Majesty the Queen in right of Canada 2009

Authors and Affiliations

  • Roxana Girju
    • 1
  • Preslav Nakov
    • 2
    • 3
  • Vivi Nastase
    • 4
  • Stan Szpakowicz
    • 5
    • 6
  • Peter Turney
    • 7
  • Deniz Yuret
    • 8
  1. 1.University of Illinois at Urbana-ChampaignUrbanaUSA
  2. 2.University of California at BerkeleyBerkeleyUSA
  3. 3.Bulgarian Academy of SciencesSofiaBulgaria
  4. 4.EML Research gGmbHHeidelbergGermany
  5. 5.University of OttawaOttawaCanada
  6. 6.Polish Academy of SciencesWarszawaPoland
  7. 7.National Research Council of CanadaOttawaCanada
  8. 8.Koç UniversitySarıyer, IstanbulTurkey

Personalised recommendations