Parameter Curation for Benchmark Queries

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8904)

Abstract

In this paper we consider the problem of generating parameters for benchmark queries so these have stable behavior despite being executed on datasets (real-world or synthetic) with skewed data distributions and value correlations. We show that uniform random sampling of the substitution parameters is not well suited for such benchmarks, since it results in unpredictable runtime behavior of queries. We present our approach of Parameter Curation with the goal of selecting parameter bindings that have consistently low-variance intermediate query result sizes throughout the query plan. Our solution is illustrated with IMDB data and the recently proposed LDBC Social Network Benchmark (SNB).

References

  1. 1.
  2. 2.
    Barahmand, S., Ghandeharizadeh, S.: BG: a benchmark to evaluate interactive social networking actions. In: CIDR (2013)Google Scholar
  3. 3.
    Boncz, P., Neumann, T., Erling, O.: TPC-H analyzed: hidden messages and lessons learned from an influential benchmark. In: Nambiar, R., Poess, M. (eds.) TPCTC 2013. LNCS, vol. 8391, pp. 61–76. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  4. 4.
    Pham, M.-D., Boncz, P., Erling, O.: S3G2: a scalable structure-correlated social graph generator. In: Nambiar, R., Poess, M. (eds.) TPCTC 2012. LNCS, vol. 7755, pp. 156–172. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  5. 5.
    Moerkotte, G.: Building query compilers. http://pi3.informatik.uni-mannheim.de/~moer/querycompiler.pdf
  6. 6.
    Poess, M., Stephens Jr., J.M.: Generating thousand benchmark queries in seconds. In: VLDB 2004, pp. 1045–1053 (2004)Google Scholar
  7. 7.
    Stephens, J.M., Poess, M.: MUDD: a multi-dimensional data generator. SIGSOFT Softw. Eng. Notes 29(1), 104–109 (2004)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.TU MunichMunichGermany
  2. 2.CWIAmsterdamThe Netherlands

Personalised recommendations