Increased Recall in Annotation Variance Detection in Treebanks

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9302)


Automatic inconsistency detection in parsed corpora is significantly helpful for building more and larger corpora of annotated texts. Inconsistencies are inevitable and originate from variance in annotation caused by different factors as, for instance, the lack of attention or the absence of clear annotation guidelines. In this paper, some results involving the automatic detection of annotation variance in parsed corpora are presented. In particular, it is shown that a generalization procedure substantially increases the recall of the variant detection algorithm proposed in [1].


Treebank Inconsistency detection Quality control 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Faria, P.: Using dominance chains to detect annotation variants in parsed corpora. In: 2014 IEEE 10th International Conference on e-Science (e-Science), vol. 2, pp. 25–32. IEEE (2014)Google Scholar
  2. 2.
    Dickinson, M., Meurers, W.D.: Detecting inconsistencies in treebanks. In: Proceedings of TLT, vol. 3, pp. 45–56 (2003)Google Scholar
  3. 3.
    Kato, Y., Matsubara, S.: Correcting errors in a treebank based on synchronous tree substitution grammar. In: Proceedings of the ACL 2010 Conference Short Papers, ACLShort 2010, pp. 74–79. Association for Computational Linguistics, Stroudsburg (2010)Google Scholar
  4. 4.
    Kulick, S., Bies, A., Mott, J.: Using derivation trees for treebank error detection. In: ACL (Short Papers), pp. 693–698 (2011)Google Scholar
  5. 5.
    Kulick, S., Bies, A., Mott, J.: Further developments in treebank error detection using derivation trees. In: Chair, N.C.C., Choukri, K., Declerck, T., Doğan, M.U., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. (eds.) Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), European Language Resources Association (ELRA), Istanbul, May 2012Google Scholar
  6. 6.
    Krasnowska, K., Przepiórkowski, A.: Detecting syntactic errors in dependency treebanks for morphosyntactically rich languages. In: Kłopotek, M.A., Koronacki, J., Marciniak, M., Mykowiecka, A., Wierzchoń, S.T. (eds.) IIS 2013. LNCS, vol. 7912, pp. 69–79. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  7. 7.
    Blaheta, D.: Handling noisy training and testing data. In: Proceedings of the ACL 2002 Conference on Empirical Methods in Natural Language Processing, EMNLP 2002, vol. 10, pp. 111–116. Association for Computational Linguistics, Stroudsburg (2002)Google Scholar
  8. 8.
    Galves, C., Faria, P.: Tycho brahe parsed corpus of historical portuguese (2010).
  9. 9.
    Marcus, M.P., Marcinkiewicz, M.A., Santorini, B.: Building a large annotated corpus of english: the penn treebank. Comput. Linguist. 19(2), 313–330 (1993)Google Scholar
  10. 10.
    Taylor, A., Marcus, M., Santorini, B.: The penn treebank: An overview. In: Abeillé, A. (eds.) Treebanks: Building and Using Syntactically Annotated Corpora, pp. 5–22. Kluwer Academic Publishers (2003)Google Scholar
  11. 11.
    Dickinson, M., Meurers, W.D.: Detecting errors in discontinuous structural annotation. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL 2005, pp. 322–329. Association for Computational Linguistics, Stroudsburg (2005)Google Scholar
  12. 12.
    Maamouri, M., Bies, A., Kulick, S., Krouna, S., Gaddeche, F., Zaghouani, W.: Arabic treebank part 3–v3.2. Linguistic Data Consortium LDC2010T08 (2010)Google Scholar
  13. 13.
    Weischedel, R., Palmer, M., Marcus, M., Hovy, E., Pradhan, S., Ramshaw, L., Xue, N., Taylor, A., Kaufman, J., Franchini, M., El-Bachouti, M., Belvin, R., Houston, A.: Ontonotes 4.0. Linguistic Data Consortium LDC2011T03 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.University of Campinas, Language Studies InstituteCampinasBrazil

Personalised recommendations