Advertisement

An Evaluation of Knowledge Base Systems for Large OWL Datasets

  • Yuanbo Guo
  • Zhengxiang Pan
  • Jeff Heflin
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3298)

Abstract

In this paper, we present an evaluation of four knowledge base systems (KBS) with respect to use in large OWL applications. To our knowledge, no experiment has been done with the scale of data used here. The smallest dataset used consists of 15 OWL files totaling 8MB, while the largest dataset consists of 999 files totaling 583MB. We evaluated two memory-based systems (OWLJessKB and memory-based Sesame) and two systems with persistent storage (database-based Sesame and DLDB-OWL). We describe how we have performed the evaluation and what factors we have considered in it. We show the results of the experiment and discuss the performance of each system. In particular, we have concluded that existing systems need to place a greater emphasis on scalability.

Keywords

Resource Description Framework Description Logic Query Response Time Test Query Knowledge Base System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alexaki, S., et al.: On Storing Voluminous RDF Description: The case of Web Portal Catalogs. In: Proc. of the 4th International Workshop on the Web and Databases (2001)Google Scholar
  2. 2.
    Dean, M., Schreiber, G. (eds.): OWL Web Ontology Language Reference, http://www.w3.org/TR/owl-ref/
  3. 3.
    Bitton, D., DeWitt, D., Turbyfill, C.: Benchmarking Database Systems, a Systematic Approach. In: Proc. of the 9th International Conference on Very Large Data Bases (1983)Google Scholar
  4. 4.
    Bitton, D., Turbyfill, C.: A Retrospective on the Wisconsin Benchmark. In: Readings in Database Systems, 2nd edn. (1994)Google Scholar
  5. 5.
    Broekstra, J., Kampman, A.: Inferencing and Truth Maintenance in RDF Schema: exploring a naive practical approach. In: Workshop on Practical and Scalable Semantic Systems, PSSS (2003)Google Scholar
  6. 6.
    Broekstra, J., Kampman, A.: Sesame: A Generic Architecture for Storing and Querying RDF and RDF Schema. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, p. 54. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  7. 7.
    Carroll, J.J., Roo, J.D. (eds.): OWL Web Ontology Test Cases, http://www.w3.org/TR/2004/REC-owl-test-20040210/
  8. 8.
    Cattell, R.G.G.: An Engineering Database Benchmark. In: Readings in Database Systems, 2nd edn. (1994)Google Scholar
  9. 9.
    Connolly, D., et al.: DAML+OIL, Reference Description (March 2001), http://www.w3.org/TR/daml+oil-reference
  10. 10.
    Elhaik, Q., Rousset, M.-C., Ycart, B.: Generating Random Benchmarks for Description Logics. In: Proc. of DL 1998 (1998)Google Scholar
  11. 11.
    Gensereth, M., Fikes, R.: Knowledge Interchange Format. Stanford Logic Report Logic-92-1, Stanford Univ., http://logic.standford.edu/kif/kif.html
  12. 12.
    Guo, Y., Heflin, J., Pan, Z.: Benchmarking DAML+OIL Repositories. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 613–627. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  13. 13.
    Guo, Y., Heflin, J., Pan, Z.: An Evaluation of Knowledge Base Systems for Large OWL Datasets. Technical report, CSE department, Lehigh University (2004) (to appear)Google Scholar
  14. 14.
    Guo, Y., Pan, Z., Heflin, J.: Choosing the Best Knowledge Base System for Large Semantic Web Applications. Poster paper at WWW 2004 (2004)Google Scholar
  15. 15.
    Haarslev, V., Moller, R.: Racer: A Core Inference Engine for the Semantic Web. In: Workshop on Evaluation on Ontology-based Tools, ISWC 2003 (2003)Google Scholar
  16. 16.
    Horrocks, I.: The FaCT System. In: de Swart, H. (ed.) TABLEAUX 1998. LNCS (LNAI), vol. 1397, p. 307. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  17. 17.
    Horrocks, I., Patel-Schneider, P.: DL Systems Comparison. In: Proc. of DL 1998 (1998)Google Scholar
  18. 18.
    Jena – A Semantic Web Framework for Java, http://jena.sourceforge.net/
  19. 19.
    Jess: the Rule Engine for the Java Platform, http://herzberg.ca.sandia.gov/jess
  20. 20.
    KAON: The KArlsruhe ONtology and Semantic Web tool suite, http://kaon.semanticweb.org/
  21. 21.
    Karvounarakis, G., et al.: Querying Community Web Portals, http://www.ics.forth.gr/proj/isst/RDF/RQL/rql.pdf
  22. 22.
    Kopena, J.B., Regli, W.C.: DAMLJessKB: A Tool for Reasoning with the Semantic Web. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 628–643. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  23. 23.
    Pan, Z., Heflin, J.: DLDB: Extending Relational Databases to Support Semantic Web Queries. In: Workshop on Practical and Scalable Semantic Systems, ISWC 2003 (2003)Google Scholar
  24. 24.
    Sintek, M., Decker, S.: TRIPLE – A Query, Inference, and Transformation Language for the Semantic Web. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, p. 364. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  25. 25.
    Stonebraker, M., et al.: The SEQUIOA 2000 Storage Benchmark. In: Readings in Database Systems, 2nd edn. (1994)Google Scholar
  26. 26.
    Tempich, C., Volz, R.: Towards a benchmark for Semantic Web reasoners–an analysis of the DAML ontology library. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, Springer, Heidelberg (2003)Google Scholar
  27. 27.
    W3C. Resource Description Framework (RDF), http://www.w3.org/RDF/

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Yuanbo Guo
    • 1
  • Zhengxiang Pan
    • 1
  • Jeff Heflin
    • 1
  1. 1.Computer Science and Engineering DepartmentLehigh UniversityBethlehemUSA

Personalised recommendations