Italian Anaphoric Annotation with the Phrase Detectives Game-with-a-Purpose

  • Livio Robaldo
  • Massimo Poesio
  • Luca Ducceschi
  • Jon Chamberlain
  • Udo Kruschwitz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6934)


Recently, web collaboration (also known as crowd sourcing) has started to emerge as a viable alternative for building the large resources that are needed to build and evaluate NLP systems.

In this spirit, the Anawiki project ( [8] aimed at experimenting with Web collaboration and human computation as a solution to the problem of creating large-scale linguistically annotated corpora. So far, the main initiative of the project has been Phrase Detectives (PD) [2], a game designed to collect judgments about anaphoric annotations. To our knowledge, Phrase Detectives was the first attempt to exploit the effort of Web volunteers to annotate corpora (subsequent efforts include [1] and [5]).


Relative Clause Dependency Tree Language Resource Annotate Corpus Dependency Parser 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Attardi, G., the Galoap Team: Phratris. Demo presented at INSEMTIVES 2010 (2010)Google Scholar
  2. 2.
    Chamberlain, J., Kruschwitz, U., Poesio, M.: Constructing an anaphorically annotated corpus with non-experts: Assessing the quality of collaborative annotations. In: Proc. of the 2009 Workshop on The People’s Web Meets NLP: Collaboratively Constructed Semantic Resources. Collocated at ACL 2009, Singapore (2009)Google Scholar
  3. 3.
    Chamberlain, J., Poesio, M., Kruschwitz, U.: Phrase detectives - a web-based collaborative annotation game. In: Proc. of I-Semantics (2008)Google Scholar
  4. 4.
    Chamberlain, J., Poesio, M., Kruschwitz, U.: A new life for a dead parrot: Incentive structures in the phrase detectives game, Madrid (2009)Google Scholar
  5. 5.
    Hladká, B., Mírovskỳ, J., Schlesinger, P.: Play the language: play coreference. In: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pp. 209–212. Association for Computational Linguistics (2009)Google Scholar
  6. 6.
    Kruschwitz, U., Chamberlain, J., Poesio, M.: (linguistic) science through web collaboration in the anawiki project, Athens (2009)Google Scholar
  7. 7.
    Lesmo, L., Lombardo, V.: Transformed subcategorization frames in chunk parsing. In: In Proc. of the 3rd Int. Conf. on Language Resources and Evaluation (LREC 2002), Las Palmas, pp. 512–519 (2002)Google Scholar
  8. 8.
    Poesio, M., Kruschwitz, U., Chamberlain, J.: Anawiki: Creating anaphorically annotated resources through web cooperation. In: Proc. of the Sixth International Language Resources and Evaluation, Marrakech, Morocco (2008)Google Scholar
  9. 9.
    Surowiecki, J.: The Wisdom of Crowds. Anchor, New York (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Livio Robaldo
    • 1
  • Massimo Poesio
    • 2
    • 4
  • Luca Ducceschi
    • 3
  • Jon Chamberlain
    • 4
  • Udo Kruschwitz
    • 4
  1. 1.University of TurinItaly
  2. 2.University of TrentoUK
  3. 3.University of Utrech/VeronaItaly
  4. 4.University of EssexUK

Personalised recommendations