A Framework for Ontology Evaluation

  • Jinie Pak
  • Lina Zhou
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 52)


The rapid growth in the number of ontologies has not met with the wide adoption of ontology in practice. Ontology evaluation can promote ontology use by facilitating the selection of a good ontology. Despite that a host of ontology evaluation methodologies are available, many of them are fragmentary and strongly tied to ontology development methodologies. Based on a review of extant ontology evaluation methods, we propose a framework for ontology evaluation. The framework provides a holistic view of ontology evaluation, suggesting both fundamental ontology dimensions and concrete criteria.


Ontology Ontology evaluation Ontology quality 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brank, J., Grobelnik, M., Mladenic, D.: A survey of ontology evaluation techniques. In: The Conference on Data Mining and Data Warehouses, SiKDD (2005)Google Scholar
  2. 2.
    Brewster, C., Alani, H., Dasmahapatra, S., Wilks, Y.: Data Driven Ontology Evaluation. In: The Language Resources and Evaluation Conference (LREC), Lisbon, Portugal (2004)Google Scholar
  3. 3.
    Daelemans, W., Reinverger, M.L.: Shallow Text Understanding for Ontology Content Evaluation. IEEE Intelligent Systems 19(4), 74–81 (2004)CrossRefGoogle Scholar
  4. 4.
    Devedzic, V.: Understanding Ontological Engineering. Communications of the ACM 45, 135–144 (2005)Google Scholar
  5. 5.
    Fernández, M., Gòmez-Pérez, A., Juristo, N.: METHONTOLOGY: From Ontological Art Towards Ontological Engineering. In: The AAAI Spring Symposium on Ontological Engineering, pp. 33–40. Stanford University, AAAI Press, Stanford, CA (1997)Google Scholar
  6. 6.
    Fernández-López, M.: Overview of Methodologies for Building Ontologies. In: The IJCAI 1999 Workshop on Ontologies and Problem-Solving Methods, Stockholm, Sweden (1999)Google Scholar
  7. 7.
    Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: A theoretical framework for ontology evaluation and validation. In: The 2nd Italian Semantic Web Workshop (SWAP) (2005)Google Scholar
  8. 8.
    Gómez-Pérez, A.: Some ideas and examples to evaluate ontologies. In: The 11th Conference on Artificial Intelligence for Applications (CAIA 1995), Washington DC, p. 299 (1995)Google Scholar
  9. 9.
    Gómez-Pérez, A.: Towards a framework to verify knowledge sharing technology. Expert Systems with Applications 4, 519–529 (1996)CrossRefGoogle Scholar
  10. 10.
    Gómez-Pérez, A.: Knowledge sharing and reuse. In: Liebowitz, J. (ed.) The Handbook of Applied Expert Systems, p. 10.1–10.35. CRC, Boca Raton (1998)Google Scholar
  11. 11.
    Gómez-Pérez, A.: Ontology Evaluation. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, pp. 251–274. Springer, Heidelberg (2003)Google Scholar
  12. 12.
    Gruber, T.R.: A translation approach to portable ontology specifications. Knowledge Acquisition 5, 199–220 (1998)CrossRefGoogle Scholar
  13. 13.
    Guarino, N., Welty, C.: Evaluating ontological decisions with OntoClean. Comm. of the ACM 45(2), 61–65 (2002)CrossRefGoogle Scholar
  14. 14.
    Hartmann, J., Sure, Y., Giboin, A., Maynard, D., Suárez-Figueroa, M., Cuel, R.: Methods for Ontology Evaluation. KWeb Deliverable D1.2.3, University of Karlsruhe (2004)Google Scholar
  15. 15.
    Kehagias, D.D., Papadimitriou, I., Hois, J., Tzovaras, D., Bateman, J.: A methodological approach for ontology evaluation and refinement. In: 2nd International ASK-IT Final Conference, Nuremberg, Germany (2008)Google Scholar
  16. 16.
    Lovrencic, S., Cubrilo, M.: Ontology Evaluation-comprising verification and validation. In: The 19th Central European Conference on Information and Intelligent Systems, pp. 657–663 (2008)Google Scholar
  17. 17.
    Lozano-Tello, A., Gómez-Pérez, A.: ONTOMETRIC: A Method to Choose the Appropriate Ontology. Journal of Database Management. Special Issue on Ontological analysis, Evaluation, and Engineering of Business Systems Analysis Methods 15(2), 1–18 (2004)Google Scholar
  18. 18.
    Maedche, A., Staab, S.: Measuring Similarity between Ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 251–263. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  19. 19.
    Noy, N.F.: Semantic Integration: A Survey Of Ontology-Based Approaches. SIGMOD Record, Special Issue on Semantic Integration 33(4), 65–70 (2004)CrossRefGoogle Scholar
  20. 20.
    Porzel, R., Malaka, R.: A Task-based Approach for Ontology Evaluation. In: ECAI 2004 Workshop on Ontology Learning and Population, Valencia, Spain (2004)Google Scholar
  21. 21.
    Spyns, P., Meersman, R., Jarrar, M.: Data modelling versus Ontology engineering. In: Sheth, A., Meersman, R. (eds.) SIGMOD Record Special Issue 31 (4), 12–17 (2002)Google Scholar
  22. 22.
    Spyns, P., Reinberger, M.-L.: Lexically evaluating ontology triples automatically generated from text. In: Gómez-Pérez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 563–577. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  23. 23.
    Dellschaft, K., Staab, S.: On How to Perform a Gold Standard Based Evaluation of Ontology Learning. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L.M. (eds.) ISWC 2006. LNCS, vol. 4273, pp. 228–241. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  24. 24.
    Stojanovic, N., Maedche, A., Staab, S., Studer, R., Sure, Y.: SEAL-A Framework for Developing Semantic PortALs. In: The 1st International Conference on Knowledge captures (ACM K-CAP), Vancouver, pp. 155–162 (2001)Google Scholar
  25. 25.
    Sure, Y., Gómez-Pérez, A., Daelemans, W., Reinberger, M., Guarino, N., Noy, N.F.: Why Evaluate Ontology Technologies? Because It Works! IEEE Intelligent Systems 19(4), 74–81 (2004)CrossRefGoogle Scholar
  26. 26.
    Uschold, M., Gruninger, M.: Ontologies: Principles, methods, and applications. Knowledge Engineering Review 11(2), 93–155 (1996)CrossRefGoogle Scholar
  27. 27.
    Ushold, M.: Creating, integrating and maintaining local and global ontologies. In: The First Workshop on Ontology Learning (OL-2000) in Conjunction with the 14th European Conference on Artificial Intelligence (ECAI ). IOS Press, Berlin (2000)Google Scholar
  28. 28.
    Welty, C.A., Guarino, N.: Supporting ontological analysis of taxonomic relationships. Data & Knowledge Engineering 39, 51–74 (2004)CrossRefGoogle Scholar
  29. 29.
    Yao, H., Orme, A.M., Etzkorn, L.: Cohesion metrics for ontology design and application. Journal of Computer Science 1(1), 107–113 (2005)CrossRefGoogle Scholar
  30. 30.
    Yu, J., Thom, J., Tam, A.: Requirements-oriented methodology for evaluating ontologies. Information Systems, Sixteenth ACM 34(8), 686–711 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jinie Pak
    • 1
  • Lina Zhou
    • 1
  1. 1.Department of Information SystemsUniversity of Maryland, Baltimore CountyBaltimoreUSA

Personalised recommendations