Abstract
Ontology evaluation poses a number of difficult challenges requiring different evaluation methodologies, particularly for a “dynamic ontology” generated by a combination of automatic and semi-automatic methods. We review evaluation methods that focus solely on syntactic (formal) correctness, on the preservation of semantic structure, or on pragmatic utility. We propose two novel methods for dynamic ontology evaluation and describe the use of these methods for evaluating the different taxonomic representations that are generated at different times or with different amounts of expert feedback. These methods are then applied to the Indiana Philosophy Ontology (InPhO), and used to guide the ontology enrichment process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Buckner, C., Niepert, M., Allen, C.: From encyclopedia to ontology: Toward dynamic representation of the discipline of philosophy. Synthese (2010)
Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modelling Ontology Evaluation and Validation. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 140–154. Springer, Heidelberg (2006)
Guarino, N., Welty, C.A.: An overview of OntoClean. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, 2nd edn., pp. 151–159. Springer, Heidelberg (2004)
Gómez-Pérez, A.: Evaluation of taxonomic knowledge in ontologies and knowledge bases. In: Proceedings of the 12th Banff Knowledge Acquisition for Knowledge-Based Systems Workshop, Banff, Alberta, Canada (1999)
Fahad, M., Qadir, M.: A Framework for Ontology Evaluation. In: Proceedings International Conference on Conceptual Structures (ICCS), Toulouse, France, pp. 7–11. Citeseer (July 2008)
Dellschaft, K., Staab, S.: Strategies for the Evaluation of Ontology Learning. In: Buitelaar, P., Cimiano, P. (eds.) Ontology Learning and Population: Bridging the Gap Between Text and Knowledge, pp. 253–272. IOS Press (2008)
Maedche, A., Staab, S.: Measuring Similarity Between Ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 251–263. Springer, Heidelberg (2002)
Brewster, C., Alani, H., Dasmahapatra, S., Wilks, Y.: Data driven ontology evaluation. In: Proceedings of LREC, vol. 2004 (2004)
Supekar, K.: A peer-review approach for ontology evaluation. In: 8th Int. Protege Conf., pp. 77–79. Citeseer (2004)
Staab, S., Gómez-Pérez, A., Daelemans, W., Reinberger, M.L., Guarino, N., Noy, N.F.: Why evaluate ontology technologies? because it works! IEEE Intelligent Systems 19, 74–81 (2004)
Lozano-Tello, A., Gómez-Pérez, A.: Ontometric: A method to choose the appropriate ontology. Journal of Database Management 15, 1–18 (2004)
Brank, J., Grobelnik, M., Mladenic, D.: Survey of ontology evaluation techniques. In: Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD) (2005)
Velardi, P., Navigli, R., Cucchiarelli, A., Neri, F.: Evaluation of OntoLearn, a methodology for automatic learning of domain ontologies. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning from Text: Methods, Evaluation and Applications. IOS Press, Amsterdam (2005)
Porzel, R., Malaka, R.: A task-based framework for ontology learning, population and evaluation. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning from Text: Methods, Evaluation and Applications. IOS Press, Amsterdam (2005)
Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human Computer Studies 43, 907–928 (1995)
Noy, N., McGuinness, D.: Ontology development 101: A guide to creating your first ontology (2001)
Niepert, M., Buckner, C., Allen, C.: A dynamic ontology for a dynamic reference work. In: Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries, p. 297. ACM (2007)
Niepert, M., Buckner, C., Allen, C.: Answer set programming on expert feedback to populate and extend dynamic ontologies. In: Proceedings of 21st FLAIRS (2008)
Smyth, P., Goodman, R.M.: An information theoretic approach to rule induction from databases. IEEE Transactions on Knowledge and Data Engineering 4, 301–316 (1992)
Shannon, C.E.: A mathematical theory of communication. University of Illinois Press, Urbana (1949)
Smith, B.: Ontology. In: Luciano, F. (ed.) Blackweel Guide to the Philosophy of Computing and Information, pp. 155–166. Blackwell, Oxford (2003)
Kuhn, T.: The Structure of Scientific Revolutions. University of Chicago Press (1962)
Jiang, J., Conrath, D.: Semantic similarity based on corpus statistics and lexical taxonomy. In: Proceedings of International Conference Research on Computational Linguistics (ROCLING X), Number Rocling X, Taiwan (1997)
Resnik, P.: Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11, 95–130 (1999)
Lin, D.: An information-theoretic definition of similarity. In: Proceedings of the 15th International Conference on Machine Learning, pp. 296–304. Citeseer (1998)
Eckert, K., Niepert, M., Niemann, C., Buckner, C., Allen, C., Stuckenschmidt, H.: Crowdsourcing the Assembly of Concept Hierarchies. In: Proceedings of the 10th ACM/IEEE Joint Conference on Digital Libraries (JCDL), Brisbane, Australia. ACM Press (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Murdock, J., Buckner, C., Allen, C. (2013). Evaluating Dynamic Ontologies. In: Fred, A., Dietz, J.L.G., Liu, K., Filipe, J. (eds) Knowledge Discovery, Knowledge Engineering and Knowledge Management. IC3K 2010. Communications in Computer and Information Science, vol 272. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29764-9_18
Download citation
DOI: https://doi.org/10.1007/978-3-642-29764-9_18
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29763-2
Online ISBN: 978-3-642-29764-9
eBook Packages: Computer ScienceComputer Science (R0)