Evaluating Dynamic Ontologies
Conference paper
Abstract
Ontology evaluation poses a number of difficult challenges requiring different evaluation methodologies, particularly for a “dynamic ontology” generated by a combination of automatic and semi-automatic methods. We review evaluation methods that focus solely on syntactic (formal) correctness, on the preservation of semantic structure, or on pragmatic utility. We propose two novel methods for dynamic ontology evaluation and describe the use of these methods for evaluating the different taxonomic representations that are generated at different times or with different amounts of expert feedback. These methods are then applied to the Indiana Philosophy Ontology (InPhO), and used to guide the ontology enrichment process.
Keywords
ontology evaluation taxonomy ontology population dynamic ontologyPreview
Unable to display preview. Download preview PDF.
References
- 1.Buckner, C., Niepert, M., Allen, C.: From encyclopedia to ontology: Toward dynamic representation of the discipline of philosophy. Synthese (2010)Google Scholar
- 2.Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modelling Ontology Evaluation and Validation. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 140–154. Springer, Heidelberg (2006)CrossRefGoogle Scholar
- 3.Guarino, N., Welty, C.A.: An overview of OntoClean. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, 2nd edn., pp. 151–159. Springer, Heidelberg (2004)Google Scholar
- 4.Gómez-Pérez, A.: Evaluation of taxonomic knowledge in ontologies and knowledge bases. In: Proceedings of the 12th Banff Knowledge Acquisition for Knowledge-Based Systems Workshop, Banff, Alberta, Canada (1999)Google Scholar
- 5.Fahad, M., Qadir, M.: A Framework for Ontology Evaluation. In: Proceedings International Conference on Conceptual Structures (ICCS), Toulouse, France, pp. 7–11. Citeseer (July 2008)Google Scholar
- 6.Dellschaft, K., Staab, S.: Strategies for the Evaluation of Ontology Learning. In: Buitelaar, P., Cimiano, P. (eds.) Ontology Learning and Population: Bridging the Gap Between Text and Knowledge, pp. 253–272. IOS Press (2008)Google Scholar
- 7.Maedche, A., Staab, S.: Measuring Similarity Between Ontologies. In: Gómez-Pérez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, pp. 251–263. Springer, Heidelberg (2002)CrossRefGoogle Scholar
- 8.Brewster, C., Alani, H., Dasmahapatra, S., Wilks, Y.: Data driven ontology evaluation. In: Proceedings of LREC, vol. 2004 (2004)Google Scholar
- 9.Supekar, K.: A peer-review approach for ontology evaluation. In: 8th Int. Protege Conf., pp. 77–79. Citeseer (2004)Google Scholar
- 10.Staab, S., Gómez-Pérez, A., Daelemans, W., Reinberger, M.L., Guarino, N., Noy, N.F.: Why evaluate ontology technologies? because it works! IEEE Intelligent Systems 19, 74–81 (2004)Google Scholar
- 11.Lozano-Tello, A., Gómez-Pérez, A.: Ontometric: A method to choose the appropriate ontology. Journal of Database Management 15, 1–18 (2004)CrossRefGoogle Scholar
- 12.Brank, J., Grobelnik, M., Mladenic, D.: Survey of ontology evaluation techniques. In: Proceedings of the Conference on Data Mining and Data Warehouses (SiKDD) (2005)Google Scholar
- 13.Velardi, P., Navigli, R., Cucchiarelli, A., Neri, F.: Evaluation of OntoLearn, a methodology for automatic learning of domain ontologies. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning from Text: Methods, Evaluation and Applications. IOS Press, Amsterdam (2005)Google Scholar
- 14.Porzel, R., Malaka, R.: A task-based framework for ontology learning, population and evaluation. In: Buitelaar, P., Cimiano, P., Magnini, B. (eds.) Ontology Learning from Text: Methods, Evaluation and Applications. IOS Press, Amsterdam (2005)Google Scholar
- 15.Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. International Journal of Human Computer Studies 43, 907–928 (1995)CrossRefGoogle Scholar
- 16.Noy, N., McGuinness, D.: Ontology development 101: A guide to creating your first ontology (2001)Google Scholar
- 17.Niepert, M., Buckner, C., Allen, C.: A dynamic ontology for a dynamic reference work. In: Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries, p. 297. ACM (2007)Google Scholar
- 18.Niepert, M., Buckner, C., Allen, C.: Answer set programming on expert feedback to populate and extend dynamic ontologies. In: Proceedings of 21st FLAIRS (2008)Google Scholar
- 19.Smyth, P., Goodman, R.M.: An information theoretic approach to rule induction from databases. IEEE Transactions on Knowledge and Data Engineering 4, 301–316 (1992)CrossRefGoogle Scholar
- 20.Shannon, C.E.: A mathematical theory of communication. University of Illinois Press, Urbana (1949)Google Scholar
- 21.Smith, B.: Ontology. In: Luciano, F. (ed.) Blackweel Guide to the Philosophy of Computing and Information, pp. 155–166. Blackwell, Oxford (2003)Google Scholar
- 22.Kuhn, T.: The Structure of Scientific Revolutions. University of Chicago Press (1962)Google Scholar
- 23.Jiang, J., Conrath, D.: Semantic similarity based on corpus statistics and lexical taxonomy. In: Proceedings of International Conference Research on Computational Linguistics (ROCLING X), Number Rocling X, Taiwan (1997)Google Scholar
- 24.Resnik, P.: Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research 11, 95–130 (1999)MATHGoogle Scholar
- 25.Lin, D.: An information-theoretic definition of similarity. In: Proceedings of the 15th International Conference on Machine Learning, pp. 296–304. Citeseer (1998)Google Scholar
- 26.Eckert, K., Niepert, M., Niemann, C., Buckner, C., Allen, C., Stuckenschmidt, H.: Crowdsourcing the Assembly of Concept Hierarchies. In: Proceedings of the 10th ACM/IEEE Joint Conference on Digital Libraries (JCDL), Brisbane, Australia. ACM Press (2010)Google Scholar
Copyright information
© Springer-Verlag Berlin Heidelberg 2013