Language Resources and Evaluation

, Volume 47, Issue 3, pp 695–721 | Cite as

Beyond sentence-level semantic role labeling: linking argument structures in discourse

  • Josef Ruppenhofer
  • Russell Lee-Goldman
  • Caroline SporlederEmail author
  • Roser Morante
Original Paper


Semantic role labeling is traditionally viewed as a sentence-level task concerned with identifying semantic arguments that are overtly realized in a fairly local context (i.e., a clause or sentence). However, this local view potentially misses important information that can only be recovered if local argument structures are linked across sentence boundaries. One important link concerns semantic arguments that remain locally unrealized (null instantiations) but can be inferred from the context. In this paper, we report on the SemEval 2010 Task-10 on “Linking Events and Their Participants in Discourse”, that addressed this problem. We discuss the corpus that was created for this task, which contains annotations on multiple levels: predicate argument structure (FrameNet and PropBank), null instantiations, and coreference. We also provide an analysis of the task and its difficulties.


SemEval Null instantiation Semantic roles Frame semantics 



We would like to thank Collin Baker, Martha Palmer and Jinho D. Choi for their collaboration on the SemEval task. We are also grateful to our annotators Markus Dräger, Lisa Fuchs, and Corinna Schorr and to the anonymous reviewers for their insightful comments and useful feedback. Josef Ruppenhofer was supported by the German Research Foundation DFG under grant PI 154/9-3 and Caroline Sporleder as part of the Cluster of Excellence Multimodal Computing and Interaction (MMCI). Roser Morante’s research was funded by the GOA project BIOGRAPH of the University of Antwerp.


  1. Baker, C., Ellsworth, M., & Erk, K. (2007). SemEval-2007 Task 19: Frame semantic structure extraction. In Proceedings of SemEval-07.Google Scholar
  2. Burchardt, A., Frank, A., & Pinkal, M. (2005). Building text meaning representations from contextually related frames—A case study. In Proceedings of IWCS-6.Google Scholar
  3. Burchardt, A., Erk, K., Frank, A., Kowalski, A., & Pado, S. (2006). Salto—A versatile multi-level annotation tool. In Proceedings of LREC 2006.Google Scholar
  4. Campbell, R. (2004). Using linguistic principles to recover empty categories. In Proceedings of the 42nd annual meeting on association for computational linguistics (pp. 645–652). Association for Computational Linguistics.Google Scholar
  5. Carreras, X., & Màrquez, L. (2004). Introduction to the CoNLL-2004 shared task: Semantic role labeling. In Proceedings of CoNLL-04 (pp. 89–97).Google Scholar
  6. Carreras, X., & Màrquez, L. (2005). Introduction to the CoNLL-2005 shared task: Semantic role labeling. In Proceedings of CoNLL-05, (pp. 152–164).Google Scholar
  7. Chen, D., Schneider, N., Das, D., & Smith, N. A. (2010). SEMAFOR: Frame argument resolution with log-linear models. In Proceedings of the 5th international workshop on semantic evaluation (pp. 264–267). Uppsala, Sweden: Association for computational linguistics.Google Scholar
  8. Collins, M. (1997). Three generative, lexicalised models for statistical parsing. In Proceedings of ACL/EACL 1997 (pp. 16–23).Google Scholar
  9. Das, D., Schneider, N., Chen, D., & Smith, N. A. (2010). Probabilistic frame-semantic parsing. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics (pp. 948–956).Google Scholar
  10. Delmonte, R. (2008) Computational linguistic text processing—Lexicon, grammar, parsing and anaphora resolution. New York: Nova Science.Google Scholar
  11. Diab, M., Alkhalifa, M., ElKateb, S., Fellbaum, C., Mansouri, A., & Palmer, M. (2007). SemEval-2007 Task 18: Arabic semantic labeling. In Proceedings of SemEval-07.Google Scholar
  12. Erk, K., & Padó, S. (2004). A powerful and versatile XML format for representing role-semantic annotation. In Proceedings of LREC-2004.Google Scholar
  13. Erk, K., & Padó, S. (2006) Shalmaneser—A flexible toolbox for semantic role assignment. In Proceedings of LREC-06.Google Scholar
  14. Fillmore, C. J. (1977). Scenes-and-frames semantics, linguistic structures processing. In A. Zampolli (Ed.), Fundamental studies in computer science, no. 59 (pp. 55–88). Amsterdam: North Holland Publishing.Google Scholar
  15. Fillmore, C. J. (1986). Pragmatically controlled zero anaphora. In Proceedings of the twelfth annual meeting of the Berkeley liguistics society.Google Scholar
  16. Fillmore, C. J., & Baker, C. F. (2001). Frame semantics for text understanding. In Proceedings of the NAACL-01 workshop on WordNet and other lexical resources.Google Scholar
  17. Gabbard, R., Marcus, M., & Kulick, S. (2006). Fully parsing the Penn Treebank. In Proceedings of the main conference on human language technology conference of the North American chapter of the association of computational linguistics (pp. 184–191). Association for Computational Linguistics.Google Scholar
  18. Gerber, M., & Chai, J. Y. (2010). Beyond NomBank: a study of implicit arguments for nominal predicates. In Proceedings of the 48th annual meeting of the association for computational linguistics, ACL ’10 (pp. 1583–1592). Stroudsburg, PA: Association for Computational Linguistics.
  19. Gildea, D., & Jurafsky, D. (2002). Automatic labeling of semantic roles. Computational Linguistics 28(3), 245–288.CrossRefGoogle Scholar
  20. Iida, R., Inui, K., & Matsumoto, Y. (2007). Zero-anaphora resolution by learning rich syntactic pattern features. ACM Transactions on Asian Language Information Processing (TALIP), 6, 1:1–1:22. ISSN 1530-0226.
  21. Litkowski, K. (2004). SENSEVAL-3 Task: Automatic labeling of semantic roles. In Proceedings of SENSEVAL-3.Google Scholar
  22. Litkowski, K. (2010). CLR: Linking events and their participants in discourse using a comprehensive FrameNet dictionary. In Proceedings of the 5th international workshop on semantic evaluation (pp. 300–303).Google Scholar
  23. Liu, H., & Singh, P. (2004). ConceptNet: A practical commonsense reasoning toolkit. BT Technology Journal 22(4), 211–226.CrossRefGoogle Scholar
  24. Màrquez, L., Villarejo, L., Martí, M. A., & Taulé, M. (2007). SemEval-2007 Task 09: Multilevel semantic annotation of Catalan and Spanish. In Proceedings of SemEval-07.Google Scholar
  25. Michaelis, L. (2006). Time and tense. In B. Aarts, & A. MacMahon (Eds.), The handbook of English linguistics, (pp. 220–234). Oxford: Blackwell.CrossRefGoogle Scholar
  26. Michaelis, L. (2010). The Oxford handbook of linguistic analysis, chapter sign-based construction grammar. Oxford: Oxford University Press.Google Scholar
  27. Morante, R., Schrauwen, S., & Daelemans, W. (2011). Annotation of negation cues and their scope guidelines v1.0. Technical Report CTR-003, CLiPS, University of Antwerp, Antwerp.Google Scholar
  28. Palmer, M. (1990). Semantic processing for finite domains. Cambridge: CUP.CrossRefGoogle Scholar
  29. Palmer, M., Dahl, D., Passonneau, R., Hirschman, L., Linebarger, M., & Dowding, J. (1986). Recovering implicit information. In Proceedings of ACL-1986.Google Scholar
  30. Palomar, M., Moreno, L., Peral, J., Muñoz, R., Ferrández, A., Martínez-Barco, P., & Saiz-Noeda, M. (2001). An algorithm for anaphora resolution in Spanish texts. Computational Linguistics, 27, 545–567. ISSN 0891-2017.Google Scholar
  31. Ruppenhofer, J., Gorinski, P., & Sporleder, C. (2011). In search of missing arguments: A linguistic approach. In Proceedings of the international conference recent advances in natural language processing 2011 (pp. 331–338).Google Scholar
  32. Ruppenhofer, J., Sporleder, C., Morante, R., Baker, C., & Palmer, M. (2009) Semeval-2010 task 10: Linking events and their participants in discourse. In The NAACL-HLT 2009 workshop on semantic evaluations: Recent achievements and future directions (SEW-09).Google Scholar
  33. Surdeanu, M., Johansson, R., Meyers, A., Màrquez, L., & Nivre, J. (2008). The CoNLL-2008 shared task on joint parsing of syntactic and semantic dependencies. In Proceedings of CoNLL-2008 (pp. 159–177).Google Scholar
  34. Tonelli, S., & Delmonte, R. (2010). Venses++: Adapting a deep semantic processing system to the identification of null instantiations. In Proceedings of the 5th international workshop on semantic evaluation (pp. 296–299). Uppsala: Association for Computational Linguistics.Google Scholar
  35. Tonelli, S., & Delmonte, R. (2011). Desperately seeking implicit arguments in text. In Proceedings of the ACL 2011 workshop on relational models of semantics (pp. 54–62).Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2012

Authors and Affiliations

  • Josef Ruppenhofer
    • 1
  • Russell Lee-Goldman
    • 2
  • Caroline Sporleder
    • 3
    Email author
  • Roser Morante
    • 4
  1. 1.University of HildesheimHildesheimGermany
  2. 2.International Computer Science InstituteBerkeleyUSA
  3. 3.Saarland UniversitySaarbrückenGermany
  4. 4.University of AntwerpAntwerpBelgium

Personalised recommendations