Italian Anaphoric Annotation with the Phrase Detectives Game-with-a-Purpose

  • Livio Robaldo
  • Massimo Poesio
  • Luca Ducceschi
  • Jon Chamberlain
  • Udo Kruschwitz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6934)


Recently, web collaboration (also known as crowd sourcing) has started to emerge as a viable alternative for building the large resources that are needed to build and evaluate NLP systems.

In this spirit, the Anawiki project ( [8] aimed at experimenting with Web collaboration and human computation as a solution to the problem of creating large-scale linguistically annotated corpora. So far, the main initiative of the project has been Phrase Detectives (PD) [2], a game designed to collect judgments about anaphoric annotations. To our knowledge, Phrase Detectives was the first attempt to exploit the effort of Web volunteers to annotate corpora (subsequent efforts include [1] and [5]).


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Attardi, G., the Galoap Team: Phratris. Demo presented at INSEMTIVES 2010 (2010)Google Scholar
  2. 2.
    Chamberlain, J., Kruschwitz, U., Poesio, M.: Constructing an anaphorically annotated corpus with non-experts: Assessing the quality of collaborative annotations. In: Proc. of the 2009 Workshop on The People’s Web Meets NLP: Collaboratively Constructed Semantic Resources. Collocated at ACL 2009, Singapore (2009)Google Scholar
  3. 3.
    Chamberlain, J., Poesio, M., Kruschwitz, U.: Phrase detectives - a web-based collaborative annotation game. In: Proc. of I-Semantics (2008)Google Scholar
  4. 4.
    Chamberlain, J., Poesio, M., Kruschwitz, U.: A new life for a dead parrot: Incentive structures in the phrase detectives game, Madrid (2009)Google Scholar
  5. 5.
    Hladká, B., Mírovskỳ, J., Schlesinger, P.: Play the language: play coreference. In: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pp. 209–212. Association for Computational Linguistics (2009)Google Scholar
  6. 6.
    Kruschwitz, U., Chamberlain, J., Poesio, M.: (linguistic) science through web collaboration in the anawiki project, Athens (2009)Google Scholar
  7. 7.
    Lesmo, L., Lombardo, V.: Transformed subcategorization frames in chunk parsing. In: In Proc. of the 3rd Int. Conf. on Language Resources and Evaluation (LREC 2002), Las Palmas, pp. 512–519 (2002)Google Scholar
  8. 8.
    Poesio, M., Kruschwitz, U., Chamberlain, J.: Anawiki: Creating anaphorically annotated resources through web cooperation. In: Proc. of the Sixth International Language Resources and Evaluation, Marrakech, Morocco (2008)Google Scholar
  9. 9.
    Surowiecki, J.: The Wisdom of Crowds. Anchor, New York (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Livio Robaldo
    • 1
  • Massimo Poesio
    • 2
    • 4
  • Luca Ducceschi
    • 3
  • Jon Chamberlain
    • 4
  • Udo Kruschwitz
    • 4
  1. 1.University of TurinItaly
  2. 2.University of TrentoUK
  3. 3.University of Utrech/VeronaItaly
  4. 4.University of EssexUK

Personalised recommendations