Abstract
Aiming to build a complete benchmark for better evaluation of existing ontology systems, we extend the well-known Lehigh University Benchmark in terms of inference and scalability testing. The extended benchmark, named University Ontology Benchmark (UOBM), includes both OWL Lite and OWL DL ontologies covering a complete set of OWL Lite and DL constructs, respectively. We also add necessary properties to construct effective instance links and improve instance generation methods to make the scalability testing more convincing. Several well-known ontology systems are evaluated on the extended benchmark and detailed discussions on both existing ontology systems and future benchmark development are presented.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Berners-Lee, T., Hendler, J., Lassila, O.: The Semantic WEB. Scientific American (2001)
Davies, J., Fensel, D., Harmelen, F. (eds.): Towards the Semantic WEB: Ontology-driven Knowledge Management. John Wiley & Sons Ltd., England (2002)
Hayes, P.: Resource Description Framework (RDF): Semantics, W3C Recommendation (2004), http://www.w3.org/TR/2004/REC-rdf-mt-20040210/#rdf_entail
Smith, M.K., Welty, C., McGuinness, D.L.: OWL Web Ontology language Guide (2004), http://www.w3.org/TR/owl-guide/
Patel-Schneider, P.F., Hayes, P., Horrocks, I.: OWL Web Ontology Language Semantics and Abstract Syntax (2004), http://www.w3.org/TR/owl-semantics/
Carey, M.J., DeWitt, D.J., Naughtor, J.F.: The 007 Benchmark. In: Proc. of ACM international conference on Management of data, vol. 22(2) (1993)
TPC Database Benchmark (2004), http://www.tpc.org/
Pan, Z., Heflin, J.: DLDB: Extending Relational Databases to Support Semantic Web Queries. In: Proc. of Workshop on Practical and Scaleable Semantic Web Systems, pp. 109–113 (2003)
Guo, Y., Pan, Z., Heflin, J.: An Evaluation of Knowledge Base Systems for Large OWL Datasets. In: Proc. of Third International Semantic Web Conference, pp. 274–288 (2004)
The Lehigh University Benchmark (2004), http://swat.cse.lehigh.edu/projects/LUBMm/index.htm
Pellet OWL Reasoner (2004), http://www.mindswap.org/2003/pellet/index.shtml
IBM Integrated Ontology Development Toolkit – Minerva (2005), http://www.alphaworks.ibm.com/tech/semanticstk
Kopena, J.B., Regli, W.C.: DAMLJessKB: A Tool for Reasoning with the Semantic Web. In: Fensel, D., Sycara, K.P., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 628–643. Springer, Heidelberg (2003)
Broekstra, J., Kampman, A.: Sesame: A Generic Architecture for Storing and Querying RDF and RDF Schema. In: Horrocks, I., Hendler, J. (eds.) ISWC 2002. LNCS, vol. 2342, p. 54. Springer, Heidelberg (2002)
Sesame. An Open Source RDF Database with Support for RDF Schema Inferencing and Querying (2002), http://www.openrdf.org/
Tempich, C., Volz, R.: Towards a benchmark for Semantic Web reasoners–an analysis of the DAML ontology library. In: Workshop on Evaluation on Ontology-based Tools, ISWC 2003 (2003)
Volz, R.: Web Ontology Reasoning with Logic Databases. Ph.D thesis, AIFB, Karlsruhe (2004)
Elhaik, Q., Rousset, M.C., Ycart, B.: Generating Random Benchmarks for Description Logics. In: Proc. of DL 1998 (1998)
Horrocks, I., Patel-Schneider, P.: DL Systems Comparison. In: Proc. of DL 1998 (1998)
Wang, S., Guo, Y., Qasem, A., Heflin, J.: Rapid Benchmarking for Semantic Web KnowledgeBase Systems, Lehigh University Technical Report LU-CSE-05-026 (2005)
OWLIM, OWL Semantic Repository (2005), http://www.ontotext.com/owlim/
Fikes, R., Hayes, P., Horrocks, I.: OWL-QL - a language for deductive query answering on the Semantic Web. J. of Web Semantics (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ma, L., Yang, Y., Qiu, Z., Xie, G., Pan, Y., Liu, S. (2006). Towards a Complete OWL Ontology Benchmark. In: Sure, Y., Domingue, J. (eds) The Semantic Web: Research and Applications. ESWC 2006. Lecture Notes in Computer Science, vol 4011. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11762256_12
Download citation
DOI: https://doi.org/10.1007/11762256_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34544-2
Online ISBN: 978-3-540-34545-9
eBook Packages: Computer ScienceComputer Science (R0)