EvalIta 2011: The Frame Labelingover Italian Texts Task

  • Roberto Basili
  • Diego De Cao
  • Alessandro Lenci
  • Alessandro Moschitti
  • Giulia Venturi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7689)


The Frame Labeling over Italian Texts (FLaIT) task held within the EvalIta 2011 challenge is here described. It focuses on the automatic annotation of free texts according to frame semantics. Systems were asked to label all semantic frames and their arguments, as evoked by predicate words occurring in plain text sentences. Proposed systems are based on a variety of learning techniques and achieve very good results, over 80% of accuracy, in most subtasks.


NLP System Evaluation Shallow Semantic Parsing Frame Semantics 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Baker, C.F., Fillmore, C.J., Lowe, J.B.: The Berkeley FrameNet Project. In: Proceedings of the 36th ACL Meeting and 17th ICCL Conference. Morgan Kaufmann (1998)Google Scholar
  2. 2.
    Fillmore, C.J.: Frame and the Semantics of Understanding. Quaderni di Semantica IV(2), 222–254 (1985)Google Scholar
  3. 3.
    Hajič, J., Ciaramita, M., Johansson, R., Kawahara, D., Martí, M.A., Màrquez, L., Meyers, A., Nivre, J., Padó, S., Štěpánek, J., Straňák, P., Surdeanu, M., Xue, N., Zhang, Y.: The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages. In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL): Shared Task, Boulder, Colorado, pp. 1–18 (June 2009)Google Scholar
  4. 4.
    Palmer, M., Gildea, D., Kingsbury, P.: The Proposition Bank: A Corpus Annotated with Semantic Roles. Computational Linguistics Journal 31(1) (2005)Google Scholar
  5. 5.
    Burchardt, A., Erk, K., Frank, A., Kowalski, A., Pado, S., Pinkal, Manfred: The SALSA corpus: a German corpus resource for lexical semantics. In: Proceedings of LREC, Genova, Italy (2006)Google Scholar
  6. 6.
    Pado, S., Lapata, M.: Cross-lingual Annotation Projection for Semantic Roles. Jouranl of Artificial Intelligence Research 36, 307–340 (2009)zbMATHGoogle Scholar
  7. 7.
    Burchardt, A., Frank, A.: Approximating Textual Entailment with LFG and FrameNet Frames. In: Proceedings of PASCAL RTE2 Workshop (2006)Google Scholar
  8. 8.
    Basili, R., De Cao, D., Croce, D., Coppola, B., Moschitti, A.: Cross-Language Frame Semantics Transfer in Bilingual Corpora. In: Gelbukh, A. (ed.) CICLing 2009. LNCS, vol. 5449, pp. 332–345. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  9. 9.
    Subirats, C., Petruck, M.: Surprise! Spanish FrameNet! In: Proc. of the Workshop on Frame Semantics at the XVII. International Congress of Linguists, Prague (2003)Google Scholar
  10. 10.
    Shen, D., Lapata, M.: Using Semantic Roles to Improve Question Answering. In: Proceedings of EMNLP-CoNLL, Prague, pp. 12–21 (2007)Google Scholar
  11. 11.
    Tonelli, S., Pianta, E.: Frame information transfer from english to italian. In: Proc. of LREC Conference, Marrakech, Marocco (2008)Google Scholar
  12. 12.
    Corazza, A., Lavelli, A., Satta, G.: Phrase-based statistical parsing. In: Proc. of EVALITA 2007 Workshop on Evaluation of NLP Tools for Italian, AI*IA (2007)Google Scholar
  13. 13.
    Lenci, A., Montemagni, S., Venturi, G., Cutrullà, M.: Enriching the ISST-TANL corpus with semantic frames. In: Proc. of LREC 2012, Instanbul (2012)Google Scholar
  14. 14.
    Bosco, C., Montemagni, S., Mazzei, A., Lombardo, V., Dell’Orletta, F., Lenci, A.: Parsing Task: comparing dependency parsers and treebanks. In: Proceedings of Evalita 2009, Reggio Emilia (2009)Google Scholar
  15. 15.
    Dell’Orletta, F., Marchi, S., Montemagni, S., Venturi, G., Agnoloni, T., Francesconi, E.: Domain Adaptation for Dependency Parsing at Evalita 2011. In: Proceedings of Evalita 2011 (2011)Google Scholar
  16. 16.
    Montemagni, S., Barsotti, F., Battista, M., Calzolari, N., Corazzari, O., Lenci, A., Zampolli, A., Fanciulli, F., Massetani, M., Raffaelli, R., Basili, R., Pazienza, M.T., Saracino, Zanzotto, F., Mana, N., Pianesi, F., Delmonte, R.: Building the Italian Syntactic–Semantic Treebank. In: Abeillé, A. (ed.) Building and Using Syntactically Annotated Corpora. Kluwer, Dordrecht (2003)Google Scholar
  17. 17.
    Testa, M., Bolioli, A., Dini, L., Mazzini, G.: Evaluation of a Semantically Oriented Dependency Grammar for Italian. In: Proc. of the EVALITA 2009 (2009)Google Scholar
  18. 18.
    Croce, D., Moschitti, A., Basili, R.: Structured Lexical Similarity via Convolution Kernels on Dependency Trees. In: Proc. of the 2011 Conf. on Empirical Methods in Natural Language Processing, Edinburgh, UK (2011)Google Scholar
  19. 19.
    Croce, D., Giannone, C., Annesi, P., Basili, R.: Towards open-domain semantic role labeling. In: Proc. of the 48th Annual Meeting of the ACL, Uppsala, Sweden (July 2010)Google Scholar
  20. 20.
    Surdeanu, M., Harabagiu, S., Williams, J., Aarseth, P.: Using Predicate-Argument Structures for Information Extraction. In: Proceedings of the Annual Meeting of the ACL (2003)Google Scholar
  21. 21.
    Moschitti, A., Pighin, D., Basili, R.: Tree Kernels for Semantic Role Labeling. Computational Linguistics 34 (2008)Google Scholar
  22. 22.
    Gildea, D., Jurafsky, D.: Automatic labeling of semantic roles. Computational Linguistics 28(3) (2002)Google Scholar
  23. 23.
    Johansson, R., Nugues, P.: The Effect of Syntactic Representation on Semantic Role Labeling. In: Proceedings of COLING (2008)Google Scholar
  24. 24.
    Johansson, R., Nugues, P.: Semantic Structure Extraction using Nonprojective Dependency Trees. In: Proceedings of SemEval 2007 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Roberto Basili
    • 1
  • Diego De Cao
    • 1
  • Alessandro Lenci
    • 2
  • Alessandro Moschitti
    • 3
  • Giulia Venturi
    • 4
  1. 1.University of Roma, Tor VergataItaly
  2. 2.University of PisaItaly
  3. 3.University of TrentoItaly
  4. 4.ILC-CNR, Scuola St. AnnaPisaItaly

Personalised recommendations