Abstract
In this paper, we present an evaluation of four knowledge base systems (KBS) with respect to use in large OWL applications. To our knowledge, no experiment has been done with the scale of data used here. The smallest dataset used consists of 15 OWL files totaling 8MB, while the largest dataset consists of 999 files totaling 583MB. We evaluated two memory-based systems (OWLJessKB and memory-based Sesame) and two systems with persistent storage (database-based Sesame and DLDB-OWL). We describe how we have performed the evaluation and what factors we have considered in it. We show the results of the experiment and discuss the performance of each system. In particular, we have concluded that existing systems need to place a greater emphasis on scalability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alexaki, S., et al.: On Storing Voluminous RDF Description: The case of Web Portal Catalogs. In: Proc. of the 4th International Workshop on the Web and Databases (2001)
Dean, M., Schreiber, G. (eds.): OWL Web Ontology Language Reference, http://www.w3.org/TR/owl-ref/
Bitton, D., DeWitt, D., Turbyfill, C.: Benchmarking Database Systems, a Systematic Approach. In: Proc. of the 9th International Conference on Very Large Data Bases (1983)
Bitton, D., Turbyfill, C.: A Retrospective on the Wisconsin Benchmark. In: Readings in Database Systems, 2nd edn. (1994)
Broekstra, J., Kampman, A.: Inferencing and Truth Maintenance in RDF Schema: exploring a naive practical approach. In: Workshop on Practical and Scalable Semantic Systems, PSSS (2003)
Broekstra, J., Kampman, A.: Sesame: A Generic Architecture for Storing and Querying RDF and RDF Schema. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, p. 54. Springer, Heidelberg (2002)
Carroll, J.J., Roo, J.D. (eds.): OWL Web Ontology Test Cases, http://www.w3.org/TR/2004/REC-owl-test-20040210/
Cattell, R.G.G.: An Engineering Database Benchmark. In: Readings in Database Systems, 2nd edn. (1994)
Connolly, D., et al.: DAML+OIL, Reference Description (March 2001), http://www.w3.org/TR/daml+oil-reference
Elhaik, Q., Rousset, M.-C., Ycart, B.: Generating Random Benchmarks for Description Logics. In: Proc. of DL 1998 (1998)
Gensereth, M., Fikes, R.: Knowledge Interchange Format. Stanford Logic Report Logic-92-1, Stanford Univ., http://logic.standford.edu/kif/kif.html
Guo, Y., Heflin, J., Pan, Z.: Benchmarking DAML+OIL Repositories. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 613–627. Springer, Heidelberg (2003)
Guo, Y., Heflin, J., Pan, Z.: An Evaluation of Knowledge Base Systems for Large OWL Datasets. Technical report, CSE department, Lehigh University (2004) (to appear)
Guo, Y., Pan, Z., Heflin, J.: Choosing the Best Knowledge Base System for Large Semantic Web Applications. Poster paper at WWW 2004 (2004)
Haarslev, V., Moller, R.: Racer: A Core Inference Engine for the Semantic Web. In: Workshop on Evaluation on Ontology-based Tools, ISWC 2003 (2003)
Horrocks, I.: The FaCT System. In: de Swart, H. (ed.) TABLEAUX 1998. LNCS (LNAI), vol. 1397, p. 307. Springer, Heidelberg (1998)
Horrocks, I., Patel-Schneider, P.: DL Systems Comparison. In: Proc. of DL 1998 (1998)
Jena – A Semantic Web Framework for Java, http://jena.sourceforge.net/
Jess: the Rule Engine for the Java Platform, http://herzberg.ca.sandia.gov/jess
KAON: The KArlsruhe ONtology and Semantic Web tool suite, http://kaon.semanticweb.org/
Karvounarakis, G., et al.: Querying Community Web Portals, http://www.ics.forth.gr/proj/isst/RDF/RQL/rql.pdf
Kopena, J.B., Regli, W.C.: DAMLJessKB: A Tool for Reasoning with the Semantic Web. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 628–643. Springer, Heidelberg (2003)
Pan, Z., Heflin, J.: DLDB: Extending Relational Databases to Support Semantic Web Queries. In: Workshop on Practical and Scalable Semantic Systems, ISWC 2003 (2003)
Sintek, M., Decker, S.: TRIPLE – A Query, Inference, and Transformation Language for the Semantic Web. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, p. 364. Springer, Heidelberg (2002)
Stonebraker, M., et al.: The SEQUIOA 2000 Storage Benchmark. In: Readings in Database Systems, 2nd edn. (1994)
Tempich, C., Volz, R.: Towards a benchmark for Semantic Web reasoners–an analysis of the DAML ontology library. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, Springer, Heidelberg (2003)
W3C. Resource Description Framework (RDF), http://www.w3.org/RDF/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Guo, Y., Pan, Z., Heflin, J. (2004). An Evaluation of Knowledge Base Systems for Large OWL Datasets. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds) The Semantic Web – ISWC 2004. ISWC 2004. Lecture Notes in Computer Science, vol 3298. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30475-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-30475-3_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23798-3
Online ISBN: 978-3-540-30475-3
eBook Packages: Springer Book Archive