Automatic Evaluation of Ontologies (AEON)

  • Johanna Völker
  • Denny Vrandečić
  • York Sure
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3729)


OntoClean is a unique approach towards the formal evaluation of ontologies, as it analyses the intensional content of concepts. Although it is well documented in numerous publications, and its importance is widely acknowledged, it is still used rather infrequently due to the high costs for applying OntoClean, especially on tagging concepts with the correct meta-properties. In order to facilitate the use of OntoClean and to enable proper evaluation of it in real-world cases, we provide AEON , a tool which automatically tags concepts with appropriate OntoClean meta-properties. The implementation can be easily expanded to check the concepts for other abstract meta-properties, thus providing for the first time tool support in order to enable intensional ontology evaluation for concepts. Our main idea is using the web as an embodiment of objective world knowledge, where we search for patterns indicating concepts meta-properties. We get an automatic tagging of the ontology, thus reducing costs tremendously. Moreover, AEON lowers the risk of having subjective taggings. As part of the evaluation we report our experiences from creating a middle-sized OntoClean-tagged reference ontology.


Automatic Evaluation Identity Criterion Negative Evidence Ontology Engineering Human Annotator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Arpírez, J.C., et al.: WebODE: a scalable workbench for ontological engineering. In: Proc. of Int. Conf. on Knowledge Capture (K-CAP), Victoria, Canada (2001)Google Scholar
  2. 2.
    Charniak, E., Berland, M.: Finding parts in very large corpora. In: Proc. of the 37th Annual Meeting of the ACL, pp. 57–64 (1999)Google Scholar
  3. 3.
    Cimiano, P., Handschuh, S., Staab, S.: Towards the self-annotating web. In: Proceedings of the 13th World Wide Web Conference, pp. 462–471 (2004)Google Scholar
  4. 4.
    Cimiano, P., Ladwig, G., Staab, S.: Gimme’ the context: Context-driven automatic semantic annotation with c-pankow. In: Proc. 14th WWW. ACM, New York (2005)Google Scholar
  5. 5.
    Etzioni, O., et al.: Web-scale information extraction in KnowItAll (preliminary results). In: Proc. 13th WWW Conf., pp. 100–109 (2004)Google Scholar
  6. 6.
    Fernández-López, M., et al.: Building a chemical ontology using Methontology and the Ontology Design Environment. IEEE Int. Systems 14(1) (January/February 1999)Google Scholar
  7. 7.
    Fernández-López, M., Gómez-Pérez, A.: The integration of ontoclean in webode. In: Proc. of the EON 2002 Workshop at 13th EKAW (2002)Google Scholar
  8. 8.
    Gangemi, A., et al.: Sweetening WordNet with Dolce. AI Magazine (Fall 2003)Google Scholar
  9. 9.
    Grefenstette, G.: The WWW as a resource for example-based MT tasks. In: Proc. of ASLIB 1999 Translating and the Computer, vol. 21 (1999)Google Scholar
  10. 10.
    Guarino, N., Welty, C.A.: A formal ontology of properties. In: Knowledge Acquisition, Modeling and Management, pp. 97–112 (2000)Google Scholar
  11. 11.
    Guarino, N., Welty, C.A.: An overview of OntoClean. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies in Inf. Sys., pp. 151–172. Springer, Heidelberg (2004)Google Scholar
  12. 12.
    Hahn, U., Schnattinger, K.: Towards text knowledge engineering. In: Proc. of AAAI 1998/IAAI 1998 (1998)Google Scholar
  13. 13.
    Hearst, M.: Automatic acquisition of hyponyms from large text corpora. In: Proc. 14th Int. Conf. on Computational Linguistics, pp. 539–545 (1992)Google Scholar
  14. 14.
    Keller, F., Lapata, M., Ourioupina, O.: Using the web to overcome data sparseness. In: Proc. of EMNLP 2002, pp. 230–237 (2002)Google Scholar
  15. 15.
    Noy, N., Fergerson, R., Musen, M.: The knowledge model of Protégé-2000: Combining interoperability and flexibility. In: Dieng, R., Corby, O. (eds.) EKAW 2000. LNCS (LNAI), vol. 1937, pp. 17–32. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  16. 16.
    Resnik, P., Smith, N.A.: The Web as a parallel corpus. Computational Linguistics 29(3), 349–380 (2003)CrossRefGoogle Scholar
  17. 17.
    Sure, Y., Angele, J., Staab, S.: OntoEdit: Multifaceted inferencing for ontology engineering. In: Spaccapietra, S., March, S., Aberer, K. (eds.) Journal on Data Semantics I. LNCS, vol. 2800, pp. 128–152. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  18. 18.
    Sure, Y., Staab, S., Studer, R.: Methodology for development and employment of ontology based knowledge management applications. SIGMOD Rec. 31(4) (2002)Google Scholar
  19. 19.
    Tempich, C., et al.: An argumentation ontology for distributed, loosely-controlled and evolving engineering processes of ontologies (DILIGENT). In: Bussler, C., et al. (eds.) ESWC 2005. LNCS, vol. 3532. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  20. 20.
    Welty, C., Mahindru, R., Chu-Carroll, J.: Evaluating ontology cleaning. In: McGuinness, D., Ferguson, G. (eds.) AAAI 2004. AAAI / MIT Press (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Johanna Völker
    • 1
  • Denny Vrandečić
    • 1
  • York Sure
    • 1
  1. 1.Institute AIFBUniversity of Karlsruhe 

Personalised recommendations