Advertisement

Corpus Annotation Pipeline for Non-standard Texts

  • Zuzana Peliknov
  • Zuzana Nevilov
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11107)

Abstract

According to some estimations (e.g. [9]), web corpora contain over 6% of foreign material (borrowings, language mixing, named entities). Since annotation pipelines are usually built upon standard and correct data, the resulting annotation of web corpora often contains serious errors.

We studied in depth annotation errors of the web corpus czTenTen 12 and proposed an extension to the tagger desamb that had been used for czTenTen annotation. First, the subcorpus was made using the most problematic documents from czTenTen. Second, measures were established for the most frequent annotation errors. Third, we established several experiments in which we extended the annotation pipeline so it could annotate foreign material and multi-word expressions. Finally, we compared the new annotations of the subcorpus with the original ones.

Keywords

Non-standard language Interlingual homographs Corpora annotation 

References

  1. 1.
    Benko, V.: Language code switching in web corpora. In: The 11th Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2017, Karlova Studanka, Czech Republic, 1–3 December 2017, pp. 97–105 (2017). http://nlp.fi.muni.cz/raslan/2017/paper11-Benko.pdf
  2. 2.
    Constant, M., Eryiit, G., Monti, J., Plas, L.V.D., Ramisch, C., Rosner, M., Todirascu, A.: Multiword expression processing: a survey. Comput. Linguist. 0(ja), 1–92 (2017).  https://doi.org/10.1162/COLI_a_00302MathSciNetCrossRefGoogle Scholar
  3. 3.
    Crystal, D.: Language and the Internet. Cambridge University Press (2006). https://books.google.cz/books?id=cnhnO0AO45AC
  4. 4.
    Jakubek, M., Kilgarriff, A., Kov, V., Rychl, P., Suchomel, V.: The TenTen Corpus Family. In: 7th International Corpus Linguistics Conference CL 2013, Lancaster, pp. 125–127 (2013). http://ucrel.lancs.ac.uk/cl2013/
  5. 5.
    Kilgarriff, A., Grefenstette, G.: Introduction to the special issue on the web as corpus. Comput. Linguist. 29(3), 333–347 (2003).  https://doi.org/10.1162/089120103322711569MathSciNetCrossRefGoogle Scholar
  6. 6.
    Kilgarriff, A., Rychl, P., Smr, P., Tugwell, D.: The sketch engine. In: Proceedings of the Eleventh EURALEX International Congress, pp. 105–116 (2004). http://www.fit.vutbr.cz/research/view_pub.php?id=7703
  7. 7.
    Nevilov, Z.: Annotation of multi-word expressions in czech texts. In: Hork, A., Rychl, P., Rambousek, A. (eds.) Ninth Workshop on Recent Advances in Slavonic Natural Language Processing, pp. 103–112. Tribun EU, Brno (2015)Google Scholar
  8. 8.
    Schfer, R.: Processing and querying large web corpora with the COW14 architecture. In: Baski, P., Biber, H., Breiteneder, E., Kupietz, M., Lngen, H., Witt, A. (eds.) Proceedings of Challenges in the Management of Large Corpora 3 (CMLC-3). IDS, Lancaster (2015). http://rolandschaefer.net/?p=749
  9. 9.
    Schfer, R., Bildhauer, F.: Web corpus construction. Synth. Lect. Hum. Lang. Technol. (2013).  https://doi.org/10.2200/S00508ED1V01Y201305HLT022CrossRefGoogle Scholar
  10. 10.
    Taul, M., et al.: Spanish treebank annotation of informal non-standard web text. In: Daniel, F., Diaz, O. (eds.) Current Trends in Web Engineering, pp. 15–27. Springer International Publishing, Cham (2015).  https://doi.org/10.1007/978-3-319-24800-4_2CrossRefGoogle Scholar
  11. 11.
    Šmerk, P.: Unsupervised learning of rules for morphological disambiguation. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2004. LNCS (LNAI), vol. 3206, pp. 211–216. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-30120-2_27CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Natural Language Processing CentreMasaryk UniversityBrnoCzech Republic

Personalised recommendations