Ontology (Network) Evaluation



Ontology evaluation refers to the activity of checking the technical quality of an ontology against a frame of reference. As such, it is of core importance for ontology engineering supporting scenarios such as ontology validation, knowledge selection, or the evaluation of knowledge extraction algorithms. In this chapter, we provide methodological guidelines for evaluating stand-alone ontologies as well as ontology networks. Our goal is not only to present the NeOn perspective on this issue but to also provide a practical outlook to the vast area of work in the area of ontology evaluation. Without performing an extensive state-of-the-art analysis of this research field, we aim to illustrate how various evaluation methods developed by the NeOn project, and not only, can be used at different stages of the evaluation process. We conclude the chapter with some concrete examples of performing ontology evaluation.


Search Task Evaluation Metrics Ontology Development Evaluation Goal Ontology Alignment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749CrossRefGoogle Scholar
  2. Alani H, Brewster C, Shadbolt N (2006) Ranking ontologies with AKTiveRank. In: 5th international Semantic Web Conference (ISWC 2006), Athens, GA, USA, pp 1–15Google Scholar
  3. Brank J, Grobelnik M, Mladenić D (2005) A survey of ontology evaluation techniques. In: Conference on Data Mining and Data Warehouses (SiKDD 2005), Ljubljana, Slovenia, pp 166–170Google Scholar
  4. Brewster C, Alani H, Dasmahapatra S, Wilks Y (2004) Data driven ontology evaluation. In: 4th international conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal, pp 164–169Google Scholar
  5. Burton-Jones A, Storey VC, Sugumaran V, Ahluwalia P (2005) A semiotic metrics suite for assessing the quality of ontologies. Data & knowledge engineering – Special issue: Natural Language and Database and Information Systems: NLDB 2003, pp 84–102Google Scholar
  6. Cantador I, Fernandez M, Castells P (2007) Improving ontology recommendation and reuse in WebCORE by Collaborative Assessments. In: Workshop on social and collaborative construction of structured knowledge at the 16th international World Wide Web conference (WWW 2007), Banff, CanadaGoogle Scholar
  7. Caracciolo C, Heguiabehere J (2009) NeOn deliverable D7.2.3. Initial network of fisheries ontologies. NeOn projectGoogle Scholar
  8. Cerbah F (2008) Learning highly structured semantic repositories from relational databases – RDBtoOnto tool. In: 5th European Semantic Web Conference (ESWC 2008), Tenerife, Spain, pp 777–781Google Scholar
  9. Cimiano P, Völker J (2005) Text2Onto – a framework for ontology learning and data-driven change discovery. In: 10th international conference on applications of Natural Language to Information Systems (NLDB-2005), Alicante, Spain, pp 227–238Google Scholar
  10. d’Aquin M, Motta E, Sabou M, Angeletou S, Gridinoc L, Lopez V, Guidi D (2008) Towards a new generation of semantic web applications. IEEE Intell Syst 23(3):20–28CrossRefGoogle Scholar
  11. d’Aquin M, Euzenat J, Duc C, Lewen H (2009) Sharing and reusing aligned ontologies with cupboard. Demo at international conference on Knowledge Capture (K-CAP 2009), Redondo Beach, CA, USAGoogle Scholar
  12. Ding L, Pan R, Finin T, Joshi A, Peng Y, Kolari P (2005) Finding and ranking knowledge on the semantic web. In: 4th international Semantic Web Conference (ISWC 2005), Galway, Ireland, pp 156–170Google Scholar
  13. Euzenat J (2007). Semantic precision and recall for ontology alignment evaluation. In: 20th international Joint Conference on Artificial Intelligence (IJCAI-2007), Hyderabad, India, pp 348–353Google Scholar
  14. Euzenat J, Shvaiko P (2007) Ontology matching. Springer, HeidelbergzbMATHGoogle Scholar
  15. Fernandez M, Cantador I, Castells P (2006) CORE: a tool for collaborative ontology reuse and evaluation. In: 4th international workshop on evaluation of ontologies for the web at the 15th international World Wide Web conference (WWW 2006), Edinburgh, ScotlandGoogle Scholar
  16. Fernandez M, Overbeeke C, Sabou M, Motta E (2009) What makes a good ontology? A case-study in fine-grained knowledge reuse. In: 4th Asian Semantic Web Conference (ASWC 2009), Shanghai, China, pp 61–75Google Scholar
  17. Guarino N, Welty C (2004) An overview of OntoClean. In: Handbook on ontologies. Springer, Berlin, pp 151–172Google Scholar
  18. Horridge M, Parsia B, Sattler U (2009) Explaining inconsistencies in OWL ontologies. In: Scalable uncertainty management. Springer, Berlin/Heidelberg, pp 124–137CrossRefGoogle Scholar
  19. Jones M, Alani H (2006) Content-based ontology ranking. In: 9th international protege conference, Stanford, CAGoogle Scholar
  20. Lewen H, Supekar K, Noy N, Musen M (2006) Topic-specific trust and open rating systems: an approach for ontology evaluation. In: 4th international workshop on Evaluation of Ontologies for the Web (EON2006) at the 15th international World Wide Web conference (WWW 2006), Edinburgh, ScotlandGoogle Scholar
  21. Lopez V, Nikolov A, Fernandez M, Sabou M, Uren V, Motta E (2009) Merging and ranking answers in the semantic web: the wisdom of crowds. In: 4th Asian Semantic Web Conference (ASWC 2009), Shanghai, China, pp 135–152Google Scholar
  22. Lopez V, Nikolov A, Sabou M, Uren V, Motta E (2010) Scaling up question-answering to linked data. In: Knowledge Engineering and Knowledge Management by the Masses (EKAW-2010), Lisbon, Portugal, pp 193–210Google Scholar
  23. Lozano-Tello A, Gómez-Pérez A (2004) Ontometric: a method to choose the appropriate ontology. J Database Manage 15(2):1–18CrossRefGoogle Scholar
  24. Maedche M, Staab S (2002) Measuring similarity between ontologies. In: 13th international conference on Knowledge Engineering and Knowledge Management (EKAW 2002), Siguenza, Spain, pp 251–263Google Scholar
  25. Patel C, Supekar K, Lee Y, Park E (2003) OntoKhoj: a semantic web Portal for ontology searching, ranking, and classification. In: 5th international workshop on Web Information and Data Management (WIDM 2003). In conjunction with the 12th international conference on Information and Knowledge Management (CIKM 2003), New Orleans, LA, USAGoogle Scholar
  26. Porzel R, Malaka R (2004) A task-based approach for ontology evaluation. In: Proceeding of ECAI 2004 workshop on ontology learning and population, Valencia, SpainGoogle Scholar
  27. Poveda-Villalón M, Suárez-Figueroa MC, Gómez-Pérez A (2009) Common pitfalls in ontology development. In: 13th Conference of the Spanish Association for Artificial Intelligence (CAEPIA 2009), Sevilla, Spain, pp 91–100Google Scholar
  28. Sabou M, Wroe C, Goble C, Mishne G (2005) Learning domain ontologies for web service descriptions: an experiment in bioinformatics. In: 14th international World Wide Web conference (WWW 2005), Chiba, Japan, pp 190–198Google Scholar
  29. Sabou M, d’Aquin M, Motta E (2008) Exploring the semantic web as background knowledge for ontology matching. J Data Semant 11:156–190Google Scholar
  30. Strasunskas D, Tomassen S (2008) Empirical insights on a value of ontology quality in ontology-driven web search. OnTheMove 2008 confederated international conferences (OTM 2008), Monterrey, Mexico, pp 1319–1337Google Scholar
  31. Suárez-Figueroa MC, Gómez-Pérez A (2008) First attempt towards a standard glossary of ontology engineering terminology. In: 8th international conference on Terminology and Knowledge Engineering (TKE 2008), Copenhagen, Demark, pp 1–15Google Scholar
  32. Suchanek FM, Kasneci G, Weikum G (2008) YAGO: a large ontology from Wikipedia and WordNet. J Web Semant 6(3):203–217CrossRefGoogle Scholar
  33. Tartir S, Arpinar I, Moore M, Sheth A, Aleman-Meza B (2005) OntoQA: metric-based ontology quality analysis. In: IEEE workshop on knowledge acquisition from distributed, autonomous, semantically heterogeneous data and knowledge sources, Houston, TXGoogle Scholar
  34. Van Hage W, Isaac A, Aleksovski Z (2007). Sample evaluation of ontology matching systems. In: 5th international workshop on Evaluation of Ontologies and Ontology-based tools (EON 2007) Located at the 6th international Semantic Web Conference (ISWC 2007), Busan, KoreaGoogle Scholar
  35. Zablith F, d’Aquin M, Sabou M, Motta E (2010) Using ontological contexts to assess the relevance of statements in ontology evolution. In: 17th conference on Knowledge Engineering and Knowledge Management by the Masses (EKAW 2010), Lisbon, Portugal, pp 226–240Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.MODUL University ViennaViennaAustria
  2. 2.Knowledge Media Institute (KMi)The Open UniversityMilton KeynesUK

Personalised recommendations