Diversified Stress Testing of RDF Data Management Systems

  • Güneş Aluç
  • Olaf Hartig
  • M. Tamer Özsu
  • Khuzaima Daudjee
Conference paper

DOI: 10.1007/978-3-319-11964-9_13

Part of the Lecture Notes in Computer Science book series (LNCS, volume 8796)
Cite this paper as:
Aluç G., Hartig O., Özsu M.T., Daudjee K. (2014) Diversified Stress Testing of RDF Data Management Systems. In: Mika P. et al. (eds) The Semantic Web – ISWC 2014. ISWC 2014. Lecture Notes in Computer Science, vol 8796. Springer, Cham

Abstract

The Resource Description Framework (RDF) is a standard for conceptually describing data on the Web, and SPARQL is the query language for RDF. As RDF data continue to be published across heterogeneous domains and integrated at Web-scale such as in the Linked Open Data (LOD) cloud, RDF data management systems are being exposed to queries that are far more diverse and workloads that are far more varied. The first contribution of our work is an in-depth experimental analysis that shows existing SPARQL benchmarks are not suitable for testing systems for diverse queries and varied workloads. To address these shortcomings, our second contribution is the Waterloo SPARQL Diversity Test Suite (WatDiv) that provides stress testing tools for RDF data management systems. Using WatDiv, we have been able to reveal issues with existing systems that went unnoticed in evaluations using earlier benchmarks. Specifically, our experiments with five popular RDF data management systems show that they cannot deliver good performance uniformly across workloads. For some queries, there can be as much as five orders of magnitude difference between the query execution time of the fastest and the slowest system while the fastest system on one query may unexpectedly time out on another query. By performing a detailed analysis, we pinpoint these problems to specific types of queries and workloads.

Keywords

RDF SPARQL systems benchmarking workload diversity 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Güneş Aluç
    • 1
  • Olaf Hartig
    • 1
  • M. Tamer Özsu
    • 1
  • Khuzaima Daudjee
    • 1
  1. 1.David R. Cheriton School of Computer ScienceWaterlooCanada

Personalised recommendations