Open Knowledge Extraction Challenge 2017

  • René SpeckEmail author
  • Michael RöderEmail author
  • Sergio OramasEmail author
  • Luis Espinosa-AnkeEmail author
  • Axel-Cyrille Ngonga NgomoEmail author
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 769)


The Open Knowledge Extraction Challenge invites researchers and practitioners from academia as well as industry to compete to the aim of pushing further the state of the art of knowledge extraction from text for the Semantic Web. The challenge has the ambition to provide a reference framework for research in this field by redefining a number of tasks typically from information and knowledge extraction by taking into account Semantic Web requirements and has the goal to test the performance of knowledge extraction systems. This year, the challenge goes in the third round and consists of three tasks which include named entity identification, typing and disambiguation by linking to a knowledge base depending on the task. The challenge makes use of small gold standard datasets that consist of manually curated documents and large silver standard datasets that consist of automatically generated synthetic documents. The performance measure of a participating system is twofold base on (1) Precision, Recall, F1-measure and on (2) Precision, Recall, F1-measure with respect to the runtime of the system.


Open Knowledge Extraction Challenge Semantic Web 



This work has been supported by the H2020 project HOBBIT (GA no. 688227) as well as the EuroStars projects DIESEL (project no. 01QE1512C) and QAMEL (project no. 01QE1549C). Also this work was partially funded by the Spanish Ministry of Economy and Competitiveness under the Maria de Maeztu Units of Excellence Programme (MDM-2015–0502).


  1. 1.
    Hellmann, S., Lehmann, J., Auer, S., Brümmer, M.: Integrating NLP using linked data. In: Alani, H., et al. (eds.) ISWC 2013. LNCS, vol. 8219, pp. 98–113. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-41338-4_7 CrossRefGoogle Scholar
  2. 2.
    Hobbs, J.: Pronoun resolution. Lingua 44, 339–352 (1978)CrossRefGoogle Scholar
  3. 3.
    Ngonga Ngomo, A.-C., Heino, N., Lyko, K., Speck, R., Kaltenböck, M.: SCMS – semantifying content management systems. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011. LNCS, vol. 7032, pp. 189–204. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25093-4_13 CrossRefGoogle Scholar
  4. 4.
    Ngonga Ngomo, A.-C., Röder, M.: HOBBIT: holistic benchmarking for big linked data. In: ESWC, EU Networking Session (2016)Google Scholar
  5. 5.
    Plu, J., Rizzo, G., Troncy, R.: A hybrid approach for entity recognition and linking. In: Gandon, F., Cabrio, E., Stankovic, M., Zimmermann, A. (eds.) SemWebEval 2015. CCIS, vol. 548, pp. 28–39. Springer, Cham (2015). doi: 10.1007/978-3-319-25518-7_3 CrossRefGoogle Scholar
  6. 6.
    Plu, J., Rizzo, G., Troncy, R.: Enhancing entity linking by combining NER models. In: Sack, H., Dietze, S., Tordai, A., Lange, C. (eds.) SemWebEval 2016. CCIS, vol. 641, pp. 17–32. Springer, Cham (2016). doi: 10.1007/978-3-319-46565-4_2 CrossRefGoogle Scholar
  7. 7.
    Plu, J., Troncy, R., Rizzo, G.: ADEL@OKE 2017: a generic method for indexing knowledge bases for entity linking. In: ESWC 2014, 14th European Semantic Web Conference, Open Extraction Challenge, 28th May-1st June 2017, Portoroz, Slovenia, May 2017Google Scholar
  8. 8.
    Röder, M., Usbeck, R., Ngonga Ngomo, A.-C.: Techreport for gerbil 1.2.2 - v1. Technical report, Leipzig University (2016)Google Scholar
  9. 9.
    Speck, R., Ngonga Ngomo, A.-C.: Ensemble learning for named entity recognition. In: Mika, P., Tudorache, T., Bernstein, A., Welty, C., Knoblock, C., Vrandečić, D., Groth, P., Noy, N., Janowicz, K., Goble, C. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 519–534. Springer, Cham (2014). doi: 10.1007/978-3-319-11964-9_33 Google Scholar
  10. 10.
    Speck, R., Ngonga Ngomo, A.-C.: Named entity recognition using fox. In: International Semantic Web Conference 2014 (ISWC2014), Demos & Posters (2014)Google Scholar
  11. 11.
    Usbeck, R., Ngonga Ngomo, A.-C., Röder, M., Gerber, D., Coelho, S.A., Auer, S., Both, A.: AGDISTIS - graph-based disambiguation of named entities using linked data. In: Mika, P., Tudorache, T., Bernstein, A., Welty, C., Knoblock, C., Vrandečić, D., Groth, P., Noy, N., Janowicz, K., Goble, C. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 457–471. Springer, Cham (2014). doi: 10.1007/978-3-319-11964-9_29 Google Scholar
  12. 12.
    Usbeck, R., Röder, M., Ngonga Ngomo, A.-C., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo, G., Sack, H., Speck, R., Troncy, R., Waitelonis, J., Wesemann, L.: GERBIL - general entity annotation benchmark framework. In: 24th WWW Conference (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.AKSW GroupUniversity of LeipzigLeipzigGermany
  2. 2.Universitat Pompeu FabraBarcelonaSpain
  3. 3.University of PaderbornPaderbornGermany

Personalised recommendations