Advertisement

Looking Back to the EVALITA Constituency Parsing Task: 2007-2011

  • Cristina Bosco
  • Alessandro Mazzei
  • Alberto Lavelli
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7689)

Abstract

The aim of the EVALITA Parsing Task (EPT) is at defining and extending Italian state-of-the-art parsing by encouraging the application of existing models and approaches, comparing paradigms and annotation formats. Therefore, in all the editions, held respectively in 2007, 2009 and 2011, the Task has been organized around two tracks, namely Dependency Parsing and Constituency Parsing, exploiting the same data sets made available by the organizers in two different formats.

This paper describes the Constituency Parsing Task assuming a historical perspective, but mainly focussing on the last edition held in 2011. It presents and compares the resources exploited for development and testing, the participant systems and the results, showing also how the resources and scores improved during the three editions of this contest.

Keywords

Constituency Parsing Evaluation Italian 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bikel, D.M.: Intricacies of Collins’ parsing model. Computational Linguistics 30(4), 479–511 (2004)zbMATHCrossRefGoogle Scholar
  2. 2.
    Black, E., Abney, S., Flickinger, D., Gdaniec, C., Grishman, R., Harrison, P., Hindle, D., Ingria, R., Jelinek, F., Klavans, J., Liberman, M., Marcus, M., Roukos, S., Santorini, B., Strzalkowski, T.: A procedure for quantitatively comparing the syntactic coverage of English. In: Proceedings of the Speech and Natural Language Workshop, Pacific Grove, CA, pp. 306–311 (1991)Google Scholar
  3. 3.
    Bod, R.: A computational model of language performance: Data oriented parsing. In: Proceedings of the 14th International Conference on Computational linguistics (CoLing 1992), Nantes, France, pp. 855–859 (1992)Google Scholar
  4. 4.
    Bos, J., Bosco, C., Mazzei, A.: Converting a dependency treebank to a Categorial Grammar treebank for Italian. In: Proceedings of the 8th Workshop on Treebanks and Linguistic Theories (TLT 2008), Milan, Italy, pp. 27–38 (2009)Google Scholar
  5. 5.
    Bosco, C.: Multiple-step treebank conversion: from dependency to Penn format. In: Proceedings of the Linguistic Annotation Workshop (LAW) 2007, Prague, pp. 164–167 (2007)Google Scholar
  6. 6.
    Bosco, C., Mazzei, A.: The EVALITA Dependency Parsing Task: From 2007 to 2011. In: Magnini, B., Cutugno, F., Falcone, M., Pianta, E. (eds.) EVALITA 2012. LNCS(LNAI), vol. 7689, pp. 1–12. Springer, Heidelberg (2012)Google Scholar
  7. 7.
    Bosco, C., Mazzei, A., Lombardo, V.: Evalita Parsing Task: an analysis of the first parsing system contest for Italian. Intelligenza Artificiale 2(IV), 30–33 (2007)Google Scholar
  8. 8.
    Bosco, C., Mazzei, A., Lombardo, V.: Evalita 2009 Parsing Task: constituency parsers and the Penn format for Italian. In: Proceedings of Evalita 2009, Reggio Emilia, Italy (2009)Google Scholar
  9. 9.
    Bosco, C., Montemagni, S., Mazzei, A., Lombardo, V., Dell’Orletta, F., Lenci, A.: Evalita 2009 Parsing Task: comparing dependency parsers and treebanks. In: Proceedings of Evalita 2009, Reggio Emilia, Italy (2009)Google Scholar
  10. 10.
    Bosco, C., Mazzei, A., Lombardo, V., Attardi, G., Corazza, A., Lavelli, A., Lesmo, L., Satta, G., Simi, M.: Comparing Italian parsers on a common treebank: the EVALITA experience. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation, LREC 2008, Marrakech, Morocco, pp. 2066–2073 (2008)Google Scholar
  11. 11.
    Collins, M.: Three generative, lexicalized models for statistical parsing. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, Madrid, Spain, pp. 16–23 (1997)Google Scholar
  12. 12.
    Corazza, A., Lavelli, A., Satta, G.: Phrase-based statistical parsing. Intelligenza Artificiale 2(IV), 38–39 (2007)Google Scholar
  13. 13.
    Corazza, A., Lavelli, A., Satta, G., Zanoli, R.: Analyzing an Italian treebank with state-of-the-art statistical parsers. In: Proceedings of the Third Workshop on Treebanks and Linguistic Theories (TLT 2004), Tübingen, Germany, pp. 39–50 (2004)Google Scholar
  14. 14.
    McClosky, D., Charniak, E., Johnson, M.: When is self-training effective for parsing? In: Proceedings of the 22th International Conference on Computational linguistics (CoLing 2008), pp. 561–568 (2008)Google Scholar
  15. 15.
    Dell’Orletta, F., Marchi, S., Montemagni, S., Venturi, G., Agnoloni, T., Francesconi, E.: Domain adaptation for dependency parsing at Evalita 2011. In: Working Notes of EVALITA 2011 (2012)Google Scholar
  16. 16.
    Klein, D., Manning, C.D.: Fast exact inference with a factored model for natural language parsing. In: Advances in Neural Information Processing Systems 15 (NIPS 2002), Vancouver, Canada (2002)Google Scholar
  17. 17.
    Klein, D., Manning, C.D.: Accurate unlexicalized parsing. In: Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Japan, pp. 423–430 (2003)Google Scholar
  18. 18.
    Lavelli, A.: The Berkeley parser at the EVALITA 2011 Constituency Parsing Task. In: Working Notes of EVALITA 2011 (2011)Google Scholar
  19. 19.
    Lavelli, A., Corazza, A.: The Berkeley Parser at the EVALITA 2009 constituency parsing task. In: Proceedings of the EVALITA 2009 Workshop on Evaluation of NLP Tools for Italian (2009)Google Scholar
  20. 20.
    Marcus, M.P., Santorini, B., Marcinkiewicz, M.A.: Building a large annotated corpus of English: The Penn Treebank. Computational Linguistics 19(2), 313–330 (1993)Google Scholar
  21. 21.
    Montemagni, S., Barsotti, F., Battista, M., Calzolari, N., Corazzari, O., Lenci, A., Zampolli, A., Fanciulli, F., Massetani, M., Raffaelli, R., Basili, R., Pazienza, M.T., Saracino, D., Zanzotto, F., Mana, N., Pianesi, F., Delmonte, R.: Building the Italian Syntactic-Semantic Treebank. In: Abeillé, A. (ed.) Building and Using Syntactically Annotated Corpora, pp. 189–210. Kluwer, Dordrecht (2003)Google Scholar
  22. 22.
    Petrov, S., Klein, D.: Improved inference for unlexicalized parsing. In: Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, Rochester, New York, pp. 404–411 (2007)Google Scholar
  23. 23.
    Petrov, S., Klein, D.: Discriminative log-linear grammars with latent variables. In: Advances in Neural Information Processing Systems 20 (NIPS 20), Vancouver, Canada, pp. 1153–1160 (2008)Google Scholar
  24. 24.
    Pianta, E.: Recovering from failure with the GraFo left corner parser. Intelligenza Artificiale 2(IV), 34–35 (2007)Google Scholar
  25. 25.
    Sangati, F.: A simple DOP model for constituency parsing of Italian sentences. In: Proceedings of the EVALITA 2009 Workshop on Evaluation of NLP Tools for Italian (October 2009)Google Scholar
  26. 26.
    Seddah, D., Tsarfaty, R., Foster, J. (eds.): Proceedings of the Second Workshop on Statistical Parsing of Morphologically Rich Languages. Association for Computational Linguistics, Dublin, Ireland (October 2011)Google Scholar
  27. 27.
    Tonelli, S., Delmonte, R., Bristot, A.: Enriching the Venice Italian Treebank with dependency and grammatical relations. In: Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), Marrakech, Morocco, pp. 1920–1924 (2008)Google Scholar
  28. 28.
    Tsarfaty, R., Seddah, D., Goldberg, Y., Kuebler, S., Versley, Y., Candito, M., Foster, J., Rehbein, I., Tounsi, L.: Statistical parsing of morphologically rich languages (SPMRL) what, how and whither. In: Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages, Los Angeles, CA, USA, pp. 1–12 (2010)Google Scholar
  29. 29.
    Venturi, G.: Design and development of TEMIS: a syntactically and semantically annotated corpus of italian legislative texts. In: Proceedings of the Workshop on Semantic Processing of Legal Texts (SPLeT 2012), pp. 1–12 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Cristina Bosco
    • 1
  • Alessandro Mazzei
    • 1
  • Alberto Lavelli
    • 2
  1. 1.Dipartimento di InformaticaUniversità di TorinoTorinoItaly
  2. 2.FBK-irstPovoItaly

Personalised recommendations