Eaglet – a Named Entity Recognition and Entity Linking Gold Standard Checking Tool

  • Kunal Jha
  • Michael Röder
  • Axel-Cyrille Ngonga Ngomo
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10577)


The desideratum to bridge the unstructured and structured data on the web has lead to the advancement of a considerable number of annotation tools and the evaluation of these Named Entity Recognition and Entity Linking systems is incontrovertibly one of the primary tasks. However, these evaluations are mostly based on manually created gold standards. As much these gold standards have an upper hand of being created by a human, it also has room for major proportion of over-sightedness. We will demonstrate Eaglet (Available at, a tool that supports the semi-automatic checking of a gold standard based on a set of uniform annotation rules.


Entity Recognition Entity Linking Benchmarks 



This work has been supported by the H2020 project HOBBIT (GA no. 688227) as well as the EuroStars projects DIESEL (project no. 01QE1512C) and QAMEL (project no. 01QE1549C).


  1. 1.
    Cornolti, M., Ferragina, P., Ciaramita, M.: A framework for benchmarking entity-annotation systems. In: Proceedings of the 22nd International Conference on World Wide Web (WWW 2013), pp. 249–260, New York. ACM (2013)Google Scholar
  2. 2.
    Jha, K., Röder, M., Ngonga Ngomo, A.-C.: All that glitters is not gold – rule-based curation of reference datasets for named entity recognition and entity linking. In: Blomqvist, E., Maynard, D., Gangemi, A., Hoekstra, R., Hitzler, P., Hartig, O. (eds.) ESWC 2017. LNCS, vol. 10249, pp. 305–320. Springer, Cham (2017). Scholar
  3. 3.
    Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S.J., McClosky, D.: The Stanford CoreNLP natural language processing toolkit. In: Association for Computational Linguistics (ACL) System Demonstrations, pp. 55–60 (2014)Google Scholar
  4. 4.
    Mendes, P.N., Jakob, M., García-Silva, A., Bizer, C.: Dbpedia spotlight: shedding light on the web of documents. In: Proceedings of the 7th International Conference on Semantic Systems, pp. 1–8. ACM (2011)Google Scholar
  5. 5.
    Michael, R., Usbeck, R., Ngonga Ngomo, A.-C.: Techreport for GERBIL 1.2.2 - V1. Technical report, Leipzig University (2016)Google Scholar
  6. 6.
    Ratinov, L., Roth, D., Downey, D., Anderson, M.: Local and global algorithms for disambiguation to Wikipedia. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 1375–1384. ACL (2011)Google Scholar
  7. 7.
    Usbeck, R., Röder, M., Ngonga Ngomo, A.-C., Baron, C., Both, A., Brümmer, M., Ceccarelli, D., Cornolti, M., Cherix, D., Eickmann, B., Ferragina, P., Lemke, C., Moro, A., Navigli, R., Piccinno, F., Rizzo, G., Sack, H., Speck, R., Troncy, R., Waitelonis, J., Wesemann, L.: GERBIL - general entity annotation benchmark framework. In: 24th WWW Conference (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Kunal Jha
    • 1
  • Michael Röder
    • 1
  • Axel-Cyrille Ngonga Ngomo
    • 1
    • 2
  1. 1.AKSW Research GroupUniversity of LeipzigLeipzigGermany
  2. 2.Data Science GroupUniversity of PaderbornPaderbornGermany

Personalised recommendations