Challenges of an Annotation Task for Open Information Extraction in Portuguese

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11122)


Open information extraction (Open IE) is a task of extracting facts from a plain text without limiting the analysis to a predefined set of relationships. Although a significant number of studies have focused on this problem in the last years, there is a lack of available linguistic resources for languages other than English. An essential resource for the evaluation of Open IE methods is notably an annotated corpus. In this work, we present the challenges involved in the creation of a golden set corpus for the Open IE task in the Portuguese language. We describe our methodology, an annotation tool to support the task and our results on performing this annotation task in a small validation corpus.


Open information extraction Portuguese Corpora Annotation 


  1. 1.
    de Abreu, S.C., Vieira, R.: Relp: Portuguese open relation extraction. Knowl. Org. 44(3), 163–177 (2017)CrossRefGoogle Scholar
  2. 2.
    Akbik, A., Löser, A.: Kraken: N-ary facts in open information extraction. In: Proceedings of AKBC-WEKEX, pp. 52–56. ACL (2012)Google Scholar
  3. 3.
    Artstein, R., Poesio, M.: Inter-coder agreement for computational linguistics. Comput. Linguist. 34(4), 555–596 (2008)CrossRefGoogle Scholar
  4. 4.
    Banko, M., Cafarella, M.J., Soderland, S., Broadhead, M., Etzioni, O.: Open information extraction from the web. Proc. IJCAI 7, 2670–2676 (2007)Google Scholar
  5. 5.
    Barbosa, G., Claro, D.B.: Utilizando features linguísticas genéricas para classificação de triplas relacionais em português (generic linguistic features for relational triples classification in portuguese)[in portuguese]. In: Proceedings of STIL, pp. 132–141 (2017)Google Scholar
  6. 6.
    Collovini, S., Machado, G., Vieira, R.: Extracting and Structuring Open Relations from Portuguese Text. In: Silva, J., Ribeiro, R., Quaresma, P., Adami, A., Branco, A. (eds.) PROPOR 2016. LNCS (LNAI), vol. 9727, pp. 153–164. Springer, Cham (2016). Scholar
  7. 7.
    Collovini, S., Pugens, L., Vanin, A.A., Vieira, R.: Extraction of relation descriptors for Portuguese using conditional random fields. In: Bazzan, A.L.C., Pichara, K. (eds.) IBERAMIA 2014. LNCS (LNAI), vol. 8864, pp. 108–119. Springer, Cham (2014). Scholar
  8. 8.
    Eugenio, B.D., Glass, M.: The kappa statistic: a second look. Comput. Linguist. 30(1), 95–101 (2004)CrossRefGoogle Scholar
  9. 9.
    Faruqui, M., Kumar, S.: Multilingual open relation extraction using cross-lingual projection. In: Proceedings of NAACL HLT, pp. 1351–1356 (2015)Google Scholar
  10. 10.
    Franco, W., Pinheiro, V., Pequeno, M., Furtado, V.: Aquisição de relações semânticas a partir de textos da wikipédia. In: Proceedings of ENIAC, p. 52. Unifor (2013)Google Scholar
  11. 11.
    Gamallo, P., Garcia, M.: Multilingual open information extraction. In: Pereira, F., Machado, P., Costa, E., Cardoso, A. (eds.) EPIA 2015. LNCS (LNAI), vol. 9273, pp. 711–722. Springer, Cham (2015). Scholar
  12. 12.
    Gamallo, P., Garcia, M.: Linguakit: uma ferramenta multilingue para a análise linguística e a extração de informação. Linguamática 9(1), 19–28 (2017)CrossRefGoogle Scholar
  13. 13.
    Gamallo, P., Garcia, M., Fernández-Lanza, S.: Dependency-based open information extraction. In: Proceedings of the Joint Workshop on Unsupervised and Semi-Supervised Learning in NLP, pp. 10–18. ROBUS-UNSUP ’12, ACL (2012)Google Scholar
  14. 14.
    Garcia, M., Gamallo, P.: Entity-centric coreference resolution of person entities for open information extraction. Procesamiento del Lenguaje Natural 53, 25–32 (2014)Google Scholar
  15. 15.
    Glauber, R., Claro, D.B.: A systematic mapping study on open information extraction. Expert Systems with Applications (2018)., Scholar
  16. 16.
    Hovy, E., Lavid, J.: Towards a ‘science’ of corpus annotation: a new methodological challenge for corpus linguistics. Int. J. Transl. 22(1), 13–36 (2010)Google Scholar
  17. 17.
    Mioto, C., Silva, M.C.F., Lopes, R.E.V.: Novo manual de sintaxe. Insular (2005).
  18. 18.
    de Oliveira, L.S., Glauber, R., Claro, D.B.: Dependentie: an open information extraction system on Portuguese by a dependence analysis. In: Proceedings of ENIAC, pp. 271–282. FC-UFU (2017)Google Scholar
  19. 19.
    Passonneau, R., Habash, N., Rambow, O.: Inter-annotator agreement on a multilingual semantic annotation task. In: Proceedings of LREC, pp. 1951–1956 (2006)Google Scholar
  20. 20.
    Pereira, V., Pinheiro, V.: Report-um sistema de extração de informações aberta para língua portuguesa (report-an open information extraction system for portuguese language). In: Proceedings of STIL, pp. 191–200 (2015)Google Scholar
  21. 21.
    Petersen, K., Vakkalanka, S., Kuzniarz, L.: Guidelines for conducting systematic mapping studies in software engineering: an update. Inf. Softw. Technol. 64, 1–18 (2015)CrossRefGoogle Scholar
  22. 22.
    Randolph, J.J.: Free-marginal multirater kappa (multirater k [free]): an alternative to fleiss’ fixed-marginal multirater kappa. Online submission (2005)Google Scholar
  23. 23.
    Rodrigues, R., Gomes, P.: Improving question-answering for Portuguese using triples extracted from corpora. In: Silva, J., Ribeiro, R., Quaresma, P., Adami, A., Branco, A. (eds.) PROPOR 2016. LNCS (LNAI), vol. 9727, pp. 25–37. Springer, Cham (2016). Scholar
  24. 24.
    Schmitz, M., Bart, R., Soderland, S., Etzioni, O., et al.: Open language learning for information extraction. In: Proceedings of EMNLP, pp. 523–534. ACL (2012)Google Scholar
  25. 25.
    Sena, C.F.L., Glauber, R., Claro, D.B.: Inference approach to enhance a Portuguese open information extraction. In: Proceedings of the 19th International Conference on Enterprise Information Systems ICEIS, vol. 1, pp. 442–451. INSTICC, ScitePress (2017)Google Scholar
  26. 26.
    Souza, E.N.P., Claro, D.B.: Extração de relações utilizando features diferenciadas para português. Linguamática 6(2), 57–65 (2014)Google Scholar
  27. 27.
    Souza, E.N.P., Claro, D.B., Glauber, R.: A similarity grammatical structures based method for improving open information systems. J. Univers. Comput. Sci. 24(1), 43–69 (2018)MathSciNetGoogle Scholar
  28. 28.
    Wu, F., Weld, D.S.: Open information extraction using wikipedia. In: Proceedings of Annual Meeting of the Association for Computational Linguistics, pp. 118–127. ACL (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Formalisms and Semantic Applications Research Group (FORMAS) LASiD/DCC/IMEFederal University of BahiaSalvadorBrazil

Personalised recommendations