The Second Open Knowledge Extraction Challenge

  • Andrea Giovanni NuzzoleseEmail author
  • Anna Lisa Gentile
  • Valentina Presutti
  • Aldo Gangemi
  • Robert Meusel
  • Heiko Paulheim
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 641)


The Open Knowledge Extraction (OKE) challenge, at its second edition, has the ambition to provide a reference framework for research on Knowledge Extraction from text for the Semantic Web by re-defining a number of tasks (typically from information and knowledge extraction), taking into account specific SW requirements. The OKE challenge defines two tasks: (1) Entity Recognition, Linking and Typing for Knowledge Base population; (2) Class Induction and entity typing for Vocabulary and Knowledge Base enrichment. Task 1 consists of identifying Entities in a sentence and create an OWL individual representing it, link to a reference KB (DBpedia) when possible and assigning a type to such individual. Task 2 consists in producing rdf:type statements, given definition texts. The participants will be given a dataset of sentences, each defining an entity (known a priori). The following systems participated to the challenge: WestLab to both Task 1 and 2, ADEL and Mannheim to Task 2 only. In this paper we describe the OKE challenge, the tasks, the datasets used for training and evaluating the systems, the evaluation method, and obtained results.


Entity Recognition Knowledge Extraction Evaluation Dataset Discourse Referent Natural Language Text 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Bizer, C., Heath, T., Berners-Lee, T.: Linked data - the story so far. Int. J. Semant. Web Inf. Syst. 5(3), 1–22 (2009)CrossRefGoogle Scholar
  2. 2.
    Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., Hellmann, S.: DBpedia - a crystallization point for the web of data. J. Web Semant. 7(3), 154–165 (2009)CrossRefGoogle Scholar
  3. 3.
    Chabchoub, M., Gagnon, M., Zouaq, A.: Collective disambiguation and semantic annotation for entity linking and typing. In: Sack et al. [14]Google Scholar
  4. 4.
    Doddington, G.R., Mitchell, A., Przybocki, M.A., Ramshaw, L.A., Strassel, S., Weischedel, R.M.: The automatic content extraction (ACE) program-tasks, data, and evaluation. In: LREC (2004)Google Scholar
  5. 5.
    Faralli, S., Ponzetto, S.P.: Open knowledge extraction challenge a hearst- like pattern-based approach to hypernym extraction and class induction. In: Sack et al. [14] (2016)Google Scholar
  6. 6.
    Gangemi, A., Guarino, N., Masolo, C., Oltramari, A., Schneider, L.: Sweetening ontologies with DOLCE. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 166–181. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Grishman, R., Sundheim, B.: Message understanding conference-6: a brief history. In: Proceedings of 16th Conference on Computational Linguistics - COLING 1996, vol. 1, pp. 466–471. Association for Computational Linguistics, Stroudsburg (1996)Google Scholar
  8. 8.
    Haidar-Ahmad, L., Font, L., Zouaq, A., Gagnon, M.: Entity typing and linking using sparql patterns and DBpedia. In: Sack et al. [14]Google Scholar
  9. 9.
    Hellmann, S., Lehmann, J., Auer, S., Brümmer, M.: Integrating NLP using linked data. In: Alani, H., et al. (eds.) ISWC 2013, Part II. LNCS, vol. 8219, pp. 98–113. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Nuzzolese, A.G., Gentile, A.L., Presutti, V., Gangemi, A., Garigliotti, D., Navigli, R.: Open knowledge extraction challenge. In: Gandon, F., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) SemWebEval 2015. CCIS, vol. 548, pp. 3–15. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-25518-7_1 CrossRefGoogle Scholar
  11. 11.
    Petasis, G., Karkaletsis, V., Paliouras, G., Krithara, A., Zavitsanos, E.: Ontology population and enrichment: state of the art. In: Paliouras, G., Spyropoulos, C.D., Tsatsaronis, G. (eds.) Bridging the Semantic Gap. LNCS, vol. 6050, pp. 134–166. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  12. 12.
    Plu, J., Rizzo, G., Troncy, R.: Enhancing entity linking by combining models. In: Sack et al. [14]Google Scholar
  13. 13.
    Röder, M., Usbeck, R., Speck, R., Ngomo, A.-C.N.: CETUS – a baseline approach to type extraction. In: Gandon, F., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) SemWebEval 2015. CCIS, vol. 548, pp. 16–27. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-25518-7_2 CrossRefGoogle Scholar
  14. 14.
    Sack, H., Dietze, S., Tordai, A., Lange, C. (eds.): The Semantic Web: ESWC Challenges, Communications in Computer and Information Science. Springer, Berlin (2016)Google Scholar
  15. 15.
    Tjong Kim Sang, E.F., Introduction to the CoNLL- shared task: language-independent named entity recognition. In: Proceedings of 6th Conference on Natural Language Learning - COLING-2002, vol. 20, pp. 1–4. Association for Computational Linguistics, Stroudsburg (2002)Google Scholar
  16. 16.
    Iordache, O.: Introduction. In: Iordache, O. (ed.) Polystochastic Models for Complexity. UCS, vol. 4, pp. 1–16. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  17. 17.
    Usbeck, R., Röder, M., Ngomo, A.N., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo, G., Sack, H., Speck, R., Troncy, R., Waitelonis, J., Wesemann, L.: GERBIL: general entity annotator benchmarking framework. In: Gangemi, A., Leonardi, S., Panconesi, A. (eds.) Proceedings of 24th International Conference on World Wide Web, WWW 2015, pp. 1133–1143. ACM (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Andrea Giovanni Nuzzolese
    • 1
    Email author
  • Anna Lisa Gentile
    • 2
  • Valentina Presutti
    • 1
  • Aldo Gangemi
    • 1
    • 3
  • Robert Meusel
    • 2
  • Heiko Paulheim
    • 2
  1. 1.Semantic Technology LaboratoryISTC-CNRRomeItaly
  2. 2.Data and Web Science GroupUniversity of MannheimMannheimGermany
  3. 3.LIPN, UMR CNRSUniversité Paris 13, Sorbone CitéParisFrance

Personalised recommendations