Application-Level Benchmarking of Big Data Systems

  • Chaitanya Baru
  • Tilmann Rabl


The increasing possibilities to collect vast amounts of data—whether in science, commerce, social networking, or government—have led to the “big data” phenomenon. The amount, rate, and variety of data that are assembled—for almost any application domain—are necessitating a reexamination of old technologies and development of new technologies to get value from the data, in a timely fashion. With increasing adoption and penetration of mobile technologies, and increasing ubiquitous use of sensors and small devices in the so-called Internet of Things, the big data phenomenon will only create more pressures on data collection and processing for transforming data into knowledge for discovery and action. A vibrant industry has been created around the big data phenomena, leading also to an energetic research agenda in this area. With the proliferation of big data hardware and software solutions in industry and research, there is a pressing need for benchmarks that can provide objective evaluations of alternative technologies and solution approaches to a given big data problem. This chapter gives an introduction to big data benchmarking and presents different proposals and standardization efforts.


Query Processing System Under Test Transaction Processing Benchmark Result Spec Benchmark 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Anon et al (1985) A measure of transaction processing power. Datamation, 1 April 1985Google Scholar
  2. 2.
    Baru C, Bhandarkar M, Poess M, Nambiar R, Rabl T (2012) Setting the direction for big data benchmark standards. In: TPC-Technical conference, VLDB 2012, 26–28 July 2012, Istanbul, TurkeyGoogle Scholar
  3. 3.
    Baru C, Bhandarkar M, Nambiar R, Poess M, Rabl T (2013) Benchmarking big data Systems and the BigData Top100 List. Big Data 1(1):60–64Google Scholar
  4. 4.
    Baru Chaitanya, Bhandarkar Milind, Nambiar Raghunath, Poess Meikel, Rabl Tilmann (2013) Benchmarking Big Data systems and the BigData Top100 List. Big Data 1(1):60–64CrossRefGoogle Scholar
  5. 5.
    Baru C, Bhandarkar M, Curino C, Danisch M, Frank M, Gowda B, Jacobsen HA, Jie H, Kumar D, Nambiar R, Poess P, Raab F, Rabl T, Ravi N, Sachs K, Sen S, Yi L, Youn C (2014) Discussion of BigBench: a proposed industry standard performance benchmark for big data. In: TPC-Technical conference, VLDBGoogle Scholar
  6. 6.
    Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen HA (2013) BigBench: towards an industry standard benchmark for big data analytics. In: Proceedings of the 2013 ACM SIGMOD conferenceGoogle Scholar
  7. 7.
    Ivanov T, Rabl T, Poess M, Queralt A, Poelman J, Poggi N (2015) Big data benchmark compendium. In: TPC technical conference, VLDB 2015, Waikoloa, Hawaii, 31 Aug 2015Google Scholar
  8. 8.
    Manyika J, Chui M, Brown B, Bughin J, Dobbs R, Roxburgh C, Byers AH (2011) Big data: the next frontier for innovation, competition, and productivity. Technical report, McKinsey Global Institute.
  9. 9.
    Rabl T, Poess M, Baru C, Jacobsen HA (2014) Specifying big data benchmarks. LNCS, vol 8163. Springer, BerlinGoogle Scholar
  10. 10.
    Rabl T, Nambiar R, Poess M, Bhandarkar M, Jacobsen HA, Baru C (2014) Advancing big data benchmarks. LNCS, vol 8585. Springer, BerlinGoogle Scholar
  11. 11.
    Shanley K (1998) History and overview of the TPC.
  12. 12.

Copyright information

© Springer India 2016

Authors and Affiliations

  1. 1.San Diego Supercomputer CenterUniversity of CaliforniaSan DiegoUSA
  2. 2.bankmarkPassauGermany

Personalised recommendations