Advertisement

Framework for an Automated Comparison of Description Logic Reasoners

  • Tom Gardiner
  • Dmitry Tsarkov
  • Ian Horrocks
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4273)

Abstract

OWL is an ontology language developed by the W3C, and although initially developed for the Semantic Web, OWL has rapidly become a de facto standard for ontology development in general. The design of OWL was heavily influenced by research in description logics, and the specification includes a formal semantics. One of the goals of this formal approach was to provide interoperability: different OWL reasoners should provide the same results when processing the same ontologies. In this paper we present a system that allows users: (a) to test and compare OWL reasoners using an extensible library of real-life ontologies; (b) to check the “correctness” of the reasoners by comparing the computed class hierarchy; (c) to compare the performance of the reasoners when performing this task; and (d) to use SQL queries to analyse and present the results in any way they see fit.

Keywords

Description Logic Benchmark Suite Ontology Language Realistic Ontology Automate Comparison 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Baader, F., Sattler, U.: An overview of tableau algorithms for description logics. Studia Logica 69(1), 5–40 (2001)MATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Bechhofer, S., Möller, R., Crowther, P.: The DIG description logic interface. In: Proceedings of DL 2003 International Workshop on Description Logics (September 2003)Google Scholar
  3. 3.
    Bechhofer, S., Volz, R., Lord, P.: Cooking the semantic web with the OWL API. In: Fensel, D., Sycara, K.P., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  4. 4.
    Carroll, J.J., De Roo, J.: OWL web ontology language test cases. W3C Recommendation (February 10, 2004), available at: http://www.w3.org/TR/owl-test/
  5. 5.
    Guo, Y., Pan, Z., Heflin, J.: An evaluation of knowledge base systems for large OWL datasets. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 274–288. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  6. 6.
    Haarslev, V., Möller, R.: High performance reasoning with very large knowledge bases: A practical case study. In: Proc. of the 17th Int. Joint Conf. on Artificial Intelligence (IJCAI 2001), pp. 161–168 (2001)Google Scholar
  7. 7.
    Haarslev, V., Möller, R.: RACER system description. In: Goré, R.P., Leitsch, A., Nipkow, T. (eds.) IJCAR 2001. LNCS, vol. 2083, pp. 701–705. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  8. 8.
    Horrocks, I.: Benchmark analysis with FaCT. In: Dyckhoff, R. (ed.) TABLEAUX 2000. LNCS, vol. 1847, pp. 62–66. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  9. 9.
    Horrocks, I., Patel-Schneider, P.F.: FaCT and DLP. In: de Swart, H. (ed.) TABLEAUX 1998. LNCS, vol. 1397, pp. 27–30. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  10. 10.
    Horrocks, I., Sattler, U., Tobies, S.: Practical reasoning for expressive description logics. In: Ganzinger, H., McAllester, D., Voronkov, A. (eds.) LPAR 1999. LNCS, vol. 1705, pp. 161–180. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  11. 11.
    Horrocks, I.: The FaCT system. In: de Swart, H. (ed.) TABLEAUX 1998. LNCS (LNAI), vol. 1397, pp. 307–312. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  12. 12.
    Horrocks, I., Patel-Schneider, P.F.: DL systems comparison. In: Proc. of the 1998 Description Logic Workshop (DL 1998). CEUR Electronic Workshop Proceedings, pp. 55–57 (1998), http://ceur-ws.org/Vol-11/
  13. 13.
    Horrocks, I., Patel-Schneider, P.F., van Harmelen, F.: From SHIQ and RDF to OWL: The making of a web ontology language. J. of Web Semantics 1(1), 7–26 (2003)Google Scholar
  14. 14.
    Horrocks, I., Sattler, U.: Ontology reasoning in the SHOQ(D) description logic. In: Proc. of the 17th Int. Joint Conf. on Artificial Intelligence (IJCAI 2001), pp. 199–204 (2001)Google Scholar
  15. 15.
    Horrocks, I., Sattler, U.: A tableaux decision procedure for SHOIQ. In: Proc. of the 19th Int. Joint Conf. on Artificial Intelligence (IJCAI 2005), pp. 448–453 (2005)Google Scholar
  16. 16.
    Hustadt, U., Schmidt, R.A.: Using resolution for testing modal satisfiability and building models. In: Gent, I.P., van Maaren, H., Walsh, T. (eds.) SAT 2000: Highlights of Satisfiability Research in the Year 2000. Frontiers in Artificial Intelligence and Applications, vol. 63. IOS Press, Amsterdam (2000), also in a special issue of Journal of Automated Reasoning (to appear)Google Scholar
  17. 17.
    Kalyanpur, A., Parsia, B., Sirin, E., Cuenca-Grau, B., Hendler, J.: SWOOP: a web ontology editing browser. J. of Web Semantics 4(2) (2005)Google Scholar
  18. 18.
    Massacci, F., Donini, F.M.: Design and results of TANCS 2000. In: Dyckhoff, R. (ed.) TABLEAUX 2000. LNCS (LNAI), vol. 1847. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  19. 19.
    Pan, Z.: Benchmarking DL reasoners using realistic ontologies. In: Proc. of the First OWL Experiences and Directions Workshop (2005)Google Scholar
  20. 20.
    Patel-Schneider, P.F.: DLP system description. In: Proc. of the 1998 Description Logic Workshop (DL 1998). CEUR Electronic Workshop Proceedings, pp. 87–89 (1998), http://ceur-ws.org/Vol-11/
  21. 21.
    Patel-Schneider, P.F., Hayes, P., Horrocks, I.: OWL web ontology language semantics and abstract syntax. W3C Recommendation (February 10, 2004), available at: http://www.w3.org/TR/owl-semantics/
  22. 22.
    Patel-Schneider, P.F., Sebastiani, R.: A new general method to generate random modal formulae for testing decision procedures. J. of Artificial Intelligence Research 18, 351–389 (2003)MATHMathSciNetGoogle Scholar
  23. 23.
    Protégé (2003), http://protege.stanford.edu/
  24. 24.
    Sirin, E., Parsia, B., Cuenca Grau, B., Kalyanpur, A., Katz, Y.: Pellet: A practical OWL-DL reasoner. Journal of Web Semantics (submitted, 2005)Google Scholar
  25. 25.
    Tsarkov, D., Horrocks, I.: Ordering heuristics for description logic reasoning. In: Proc. of the 19th Int. Joint Conf. on Artificial Intelligence (IJCAI 2005), pp. 609–614 (2005)Google Scholar
  26. 26.
    Tsarkov, D., Horrocks, I.: FaCT++ description logic reasoner: System description. In: Furbach, U., Shankar, N. (eds.) IJCAR 2006. LNCS, vol. 4130, pp. 292–297. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  27. 27.
    Tsarkov, D., Riazanov, A., Bechhofer, S., Horrocks, I.: Using Vampire to reason with OWL. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 471–485. Springer, Heidelberg (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Tom Gardiner
    • 1
  • Dmitry Tsarkov
    • 1
  • Ian Horrocks
    • 1
  1. 1.University of ManchesterManchesterUK

Personalised recommendations