Skip to main content

Application-Level Benchmarking of Big Data Systems

  • Chapter
  • First Online:
Big Data Analytics

Abstract

The increasing possibilities to collect vast amounts of data—whether in science, commerce, social networking, or government—have led to the “big data” phenomenon. The amount, rate, and variety of data that are assembled—for almost any application domain—are necessitating a reexamination of old technologies and development of new technologies to get value from the data, in a timely fashion. With increasing adoption and penetration of mobile technologies, and increasing ubiquitous use of sensors and small devices in the so-called Internet of Things, the big data phenomenon will only create more pressures on data collection and processing for transforming data into knowledge for discovery and action. A vibrant industry has been created around the big data phenomena, leading also to an energetic research agenda in this area. With the proliferation of big data hardware and software solutions in industry and research, there is a pressing need for benchmarks that can provide objective evaluations of alternative technologies and solution approaches to a given big data problem. This chapter gives an introduction to big data benchmarking and presents different proposals and standardization efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    The term “big data” is often written with capitals, i.e. Big Data. In this paper, we have chosen to write this term without capitalization.

  2. 2.

    NIST Big Data Public Working Group, http://bigdatawg.nist.gov/home.php.

  3. 3.

    NIST NBD-PWG Use Cases and Requirement, http://bigdatawg.nist.gov/usecases.php.

  4. 4.

    Genome in a Bottle Consortium, https://sites.stanford.edu/abms/giab.

  5. 5.

    TPC, http://www.tpc.org/.

  6. 6.

    http://www.slideshare.net/tilmann_rabl/ieee2014-tutorialbarurabl?qid=d21f7949-d467-4824-adb1-2e394e9a2239&v=&b=&from_search=1.

  7. 7.

    SPEC RG big data - http://research.spec.org/working-groups/big-data-working-group.

  8. 8.

    BigData Top100 - http://www.bigdatatop100.org/.

References

  1. Anon et al (1985) A measure of transaction processing power. Datamation, 1 April 1985

    Google Scholar 

  2. Baru C, Bhandarkar M, Poess M, Nambiar R, Rabl T (2012) Setting the direction for big data benchmark standards. In: TPC-Technical conference, VLDB 2012, 26–28 July 2012, Istanbul, Turkey

    Google Scholar 

  3. Baru C, Bhandarkar M, Nambiar R, Poess M, Rabl T (2013) Benchmarking big data Systems and the BigData Top100 List. Big Data 1(1):60–64

    Google Scholar 

  4. Baru Chaitanya, Bhandarkar Milind, Nambiar Raghunath, Poess Meikel, Rabl Tilmann (2013) Benchmarking Big Data systems and the BigData Top100 List. Big Data 1(1):60–64

    Article  Google Scholar 

  5. Baru C, Bhandarkar M, Curino C, Danisch M, Frank M, Gowda B, Jacobsen HA, Jie H, Kumar D, Nambiar R, Poess P, Raab F, Rabl T, Ravi N, Sachs K, Sen S, Yi L, Youn C (2014) Discussion of BigBench: a proposed industry standard performance benchmark for big data. In: TPC-Technical conference, VLDB

    Google Scholar 

  6. Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen HA (2013) BigBench: towards an industry standard benchmark for big data analytics. In: Proceedings of the 2013 ACM SIGMOD conference

    Google Scholar 

  7. Ivanov T, Rabl T, Poess M, Queralt A, Poelman J, Poggi N (2015) Big data benchmark compendium. In: TPC technical conference, VLDB 2015, Waikoloa, Hawaii, 31 Aug 2015

    Google Scholar 

  8. Manyika J, Chui M, Brown B, Bughin J, Dobbs R, Roxburgh C, Byers AH (2011) Big data: the next frontier for innovation, competition, and productivity. Technical report, McKinsey Global Institute. http://www.mckinsey.com/insights/mgi/research/technology_and_innovation/big_data_the_next_frontier_for_innovation

  9. Rabl T, Poess M, Baru C, Jacobsen HA (2014) Specifying big data benchmarks. LNCS, vol 8163. Springer, Berlin

    Google Scholar 

  10. Rabl T, Nambiar R, Poess M, Bhandarkar M, Jacobsen HA, Baru C (2014) Advancing big data benchmarks. LNCS, vol 8585. Springer, Berlin

    Google Scholar 

  11. Shanley K (1998) History and overview of the TPC. http://www.tpc.org/information/about/history.asp

  12. Serlin O. The History of DebitCredit and the TPC, http://research.microsoft.com/en-us/um/people/gray/benchmarkhandbook/chapter2.pdf

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Chaitanya Baru .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer India

About this chapter

Cite this chapter

Baru, C., Rabl, T. (2016). Application-Level Benchmarking of Big Data Systems. In: Pyne, S., Rao, B., Rao, S. (eds) Big Data Analytics. Springer, New Delhi. https://doi.org/10.1007/978-81-322-3628-3_10

Download citation

Publish with us

Policies and ethics