Semantic Web Evaluation Challenge

Semantic Web Evaluation Challenges pp 3-15 | Cite as

Open Knowledge Extraction Challenge

  • Andrea Giovanni Nuzzolese
  • Anna Lisa Gentile
  • Valentina Presutti
  • Aldo Gangemi
  • Darío Garigliotti
  • Roberto Navigli
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 548)

Abstract

The Open Knowledge Extraction (OKE) challenge is aimed at promoting research in the automatic extraction of structured content from textual data and its representation and publication as Linked Data. We designed two extraction tasks: (1) Entity Recognition, Linking and Typing and (2) Class Induction and entity typing. The challenge saw the participations of four systems: CETUS-FOX and FRED participating to both tasks, Adel participating to Task 1 and OAK@Sheffield participating to Task 2. In this paper we describe the OKE challenge, the tasks, the datasets used for training and evaluating the systems, the evaluation method, and obtained results.

References

  1. 1.
    Bizer, C., Heath, T., Berners-Lee, T.: Linked data - the story so far. Int. J. Semantic Web Inf. Syst. 5(3), 1–22 (2009)CrossRefGoogle Scholar
  2. 2.
    Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., Hellmann, S.: DBpedia - a crystallization point for the web of data. J. Web Sem. 7(3), 154–165 (2009)CrossRefGoogle Scholar
  3. 3.
    Consoli, S., Reforgiato, D.: Using fred for named entity resolution, linking and typing for knowledge base population. In: Gandon, F., Sabou, M., Sack, H., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) ESWC 2015 Challenges, CCIS, pp. 40–50. Springer International Publishing, Switzerland (2015)Google Scholar
  4. 4.
    Doddington, G.R., Mitchell, A., Przybocki, M.A., Ramshaw, L.A., Strassel, S., Weischedel, R.M.: The automatic content extraction (ace) program-tasks, data, and evaluation. In: LREC (2004)Google Scholar
  5. 5.
    Gangemi, A., Guarino, N., Masolo, C., Oltramari, A., Schneider, L.: Sweetening ontologies with DOLCE. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 166–181. Springer, Heidelberg (2002) CrossRefGoogle Scholar
  6. 6.
    Gao, J., Mazumdar, S.: Exploiting linked open data to uncover entity types. In: Gandon, F., Sabou, M., Sack, H., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) ESWC 2015 Challenges, CCIS, pp. 51–62. Springer International Publishing, Switzerland (2015)Google Scholar
  7. 7.
    Grishman, R., Sundheim, B.: Message understanding conference-6: a brief history. In: Proceedings of the 16th Conference on Computational Linguistics, COLING 1996, vol. 1, pp. 466–471. Association for Computational Linguistics, Stroudsburg, PA, USA (1996)Google Scholar
  8. 8.
    Hellmann, S., Lehmann, J., Auer, S., Brümmer, M.: Integrating NLP using linked data. In: Alani, H., Kagal, L., Fokoue, A., Groth, P., Biemann, C., Parreira, J.X., Aroyo, L., Noy, N., Welty, C., Janowicz, K. (eds.) ISWC 2013, Part II. LNCS, vol. 8219, pp. 98–113. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  9. 9.
    Petasis, G., Karkaletsis, V., Paliouras, G., Krithara, A., Zavitsanos, E.: Ontology population and enrichment: state of the art. In: Paliouras, G., Spyropoulos, C.D., Tsatsaronis, G. (eds.) Multimedia Information Extraction. LNCS, vol. 6050, pp. 134–166. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  10. 10.
    Plu, J., Rizzo, G., Troncy, R.: A hybrid approach for entity recognition and linking. In: Gandon, F., Sabou, M., Sack, H., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) ESWC 2015 Challenges, CCIS, pp. 28–39. Springer International Publishing, Switzerland (2015)Google Scholar
  11. 11.
    Röder, M., Usbeck, R., Ngonga Ngomo, A.-C.: Cetus - a baseline approach to type extraction. In: Gandon, F., Sabou, M., Sack, H., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) ESWC 2015 Challenges, CCIS, pp. 16–27. Springer International Publishing, Switzerland (2015)Google Scholar
  12. 12.
    Tjong Kim Sang, E.F.: Introduction to the conll-2002 shared task: language-independent named entity recognition. In: Proceedings of the 6th Conference on Natural Language Learning, COLING-02, vol. 20, pp. 1–4. Association for Computational Linguistics, Stroudsburg, PA, USA (2002)Google Scholar
  13. 13.
    Iordache, O.: Introduction. In: Iordache, O. (ed.) Polystochastic Models for Complexity. UCS, vol. 4, pp. 1–16. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  14. 14.
    Usbeck, R., Röder, M., Ngomo, A.N., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo, G., Sack, H., Speck, R., Troncy, R., Waitelonis, J., Wesemann, L.: GERBIL: general entity annotator benchmarking framework. In Gangemi, A., Leonardi, S., Panconesi, A. (eds.) Proceedings of the 24th International Conference on World Wide Web, WWW 2015, pp. 1133–1143. ACM (2015)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Andrea Giovanni Nuzzolese
    • 1
  • Anna Lisa Gentile
    • 2
  • Valentina Presutti
    • 1
  • Aldo Gangemi
    • 1
  • Darío Garigliotti
    • 3
  • Roberto Navigli
    • 3
  1. 1.Semantic Technology Lab, ISTC-CNRRomeItaly
  2. 2.Department of Computer ScienceUniversity of SheffieldSheffieldUK
  3. 3.Department of Computer ScienceSapienza University of RomeRomeItaly

Personalised recommendations