Technology Conference on Performance Evaluation and Benchmarking

TPCTC 2014: Performance Characterization and Benchmarking. Traditional to Big Data pp 113-129

Parameter Curation for Benchmark Queries

Conference paper

DOI: 10.1007/978-3-319-15350-6_8

Volume 8904 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Gubichev A., Boncz P. (2015) Parameter Curation for Benchmark Queries. In: Nambiar R., Poess M. (eds) Performance Characterization and Benchmarking. Traditional to Big Data. TPCTC 2014. Lecture Notes in Computer Science, vol 8904. Springer, Cham

Abstract

In this paper we consider the problem of generating parameters for benchmark queries so these have stable behavior despite being executed on datasets (real-world or synthetic) with skewed data distributions and value correlations. We show that uniform random sampling of the substitution parameters is not well suited for such benchmarks, since it results in unpredictable runtime behavior of queries. We present our approach of Parameter Curation with the goal of selecting parameter bindings that have consistently low-variance intermediate query result sizes throughout the query plan. Our solution is illustrated with IMDB data and the recently proposed LDBC Social Network Benchmark (SNB).

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.TU MunichMunichGermany
  2. 2.CWIAmsterdamThe Netherlands