Encyclopedia of Database Systems

2018 Edition
| Editors: Ling Liu, M. Tamer Özsu

Database Benchmarks

  • Philippe Bonnet
  • Dennis Shasha
Reference work entry
DOI: https://doi.org/10.1007/978-1-4614-8265-9_80797

Definition

A database benchmark is a reproducible experimental framework for characterizing and comparing the performance (time, memory, or quality) of database systems or algorithms on those systems. Such an experimental framework defines the system under test, the workload, metrics, and experiments.

The system under test comprises a database system and its execution environment, composed of operating system services on top of hardware components. The benchmark may define constraints for the execution environment to ensure that the performance of different database systems is comparable.

The workload typically consists of a synthetic data set, which is either given explicitly as a file or defined by intension with a data generation algorithm. For instance, a relational database benchmarks may rely on a well-defined schema with data distribution rules for generating attribute values as well as a scale factor to create instances of various sizes. An XML database benchmarks may define a...

This is a preview of subscription content, log in to check access.

Recommended Reading

  1. 1.
    Angles R, Boncz P, Larriba-Pey J, Fundulaki I, Neumann T, Erling O, Neubauer P, Martinez-Bazan N, Kotsev V, Toma I. The linked data benchmark council: a graph and RDF industry benchmarking effort. ACM SIGMOD Rec. 2014;43(1):27–31.CrossRefGoogle Scholar
  2. 2.
    Anon et al, Bitton D, Brown M, Catell R, Ceri S, Chou T, DeWitt D, Gawlick D, Garcia-Molina H, Good B, Gray J, Homan P, Jolls B, Lukes T, Lazowska E, Nauman J, Pong M, Spector A, Trieber K, Sammer H, Serlin O, Stonebraker M, Reuter A, Weinberger P. A measure of transaction processing power. Datamation. 1985;31(7):112–8.Google Scholar
  3. 3.
    DeWitt DJ, Levine C. Not just correct, but correct and fast: a look at one of Jim Gray’s contributions to database system performance. ACM SIGMOD Rec. 2008;37(2):45–9.CrossRefGoogle Scholar
  4. 4.
    Gray J. Benchmark handbook: for database and transaction processing systems. San Francisco: Morgan Kaufmann; 1992. http://research.microsoft.com/en-us/um/people/gray/BenchmarkHandbook/TOC.htmzbMATHGoogle Scholar
  5. 5.
    Schmidt A, Waas F, Kersten M, Florescu D, Carey MJ, Manolescu I, Busse R. Why and how to benchmark XML databases. ACM SIGMOD Rec. 2001;30(3):27–32.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Computer ScienceIT University of CopenhagenCopenhagenDenmark
  2. 2.Department of Computer ScienceNew York UniversityNew YorkUSA

Section editors and affiliations

  • Dennis Shasha
    • 1
  1. 1.Department of Computer ScienceNew York UniversityNew YorkUSA