Survey on Big Data and Decision Support Benchmarks

  • Melyssa Barata
  • Jorge Bernardino
  • Pedro Furtado
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8645)


Benchmarking is a common practice for the evaluation of database computer systems. By executing certain benchmarks, manufacturers and researchers are able to highlight the characteristics of a certain system and are also able to rank the system against the rest. On the other hand, at the moment, BigData is a hot topic. It concerns dealing efficiently with information that is challenging to handle, due to volume, velocity or variety. As more and more platforms are proposed to deal with BigData, it becomes important to have benchmarks that can be used to evaluate performance characteristics of such platforms. At the same time, Decision Support applications are related to BigData, as they need to efficiently deal with huge datasets. In this paper we describe benchmarks representing Decision Support Systems (TPC-H, SSB, TPC-DS), and benchmarks for the Big Data class (YCSB, BigBench, and BigFrame), in order to help users to choose the most appropriate one for their needs. We also characterize the relationship between Big Data benchmarks and Decision Support benchmarks.


Benchmarking Decision Support Systems Big Data YCSB BigBench BigFrame TPC-H SSB TPC-DS 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    5 Steps To Benchmarking Managed NoSQL - DynamoDB Vs Cassandra, (accessed November 2, 2013)
  2. 2.
    Bog, A.: Benchmarking Transaction and Analytical Processing Systems. In: The Creation of a Mixed Workload Benchmark and its Application. Springer (2009) ISBN: 978-3-642-38069-3Google Scholar
  3. 3.
    Ghazal, A., Rabl, T., Hu, M., Raab, F., Poess, M., Croletteand, A., Jacobsen, H.A.: BigBench: Towards an Industry Standard Benchmark for Big Data Analytics. In: SIGMOD Conference, pp. 1197–1208 (2013)Google Scholar
  4. 4.
    Thanopoulou, A., Carreira, P., Galhardas, H.: Benchmarking with TPC-H on Off-the-Shelf Hardware. In: ICEIS (1), pp. 205–208 (2012)Google Scholar
  5. 5.
    An overview of the TPC-H benchmark on HP ProLiant servers and server blades, (accessed October 14, 2013)
  6. 6.
    Cooper, B.F., Silberstein, A., Tam, E., Ramakrishnan, R., Sears, R.: Benchmarking Cloud Serving Systems with YCSB. In: SoCC, pp. 143–154 6 (2010)Google Scholar
  7. 7.
    Big Data Insights, (accessed November 1, 2013)
  8. 8.
    Big Data, (accessed December 10, 2013)
  9. 9.
    BigFrame User Guide, (accessed November 1, 2013)
  10. 10.
    Decision Support, (accessed December 10, 2013)
  11. 11.
    Li, F., Ooi, B.C., Ozsu, M.T., Wu, S.: Distributed Data Management Using MapReduce. ACM Comput. Surv. 46(3), 31 (2014)Google Scholar
  12. 12.
    Poess, M., Nambiar, R.O., Walrath, D.: Why You Should Run TPC-DS:A Workload Analysis. In: VLDB, pp. 1138–1149 (2007)Google Scholar
  13. 13.
    O’Neil, P., O’Neil, E., Chen, X.: The star schema benchmark (SSB). University of Massachusetts, Boston, Technical Report (2007)Google Scholar
  14. 14.
    TPC-DS, (accessed October 12, 2013)
  15. 15.
    TPC-DS, (accessed October 12, 2013), 4
  16. 16.
    TPC-H Benchmark, (accessed October 18, 2013)
  17. 17.

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Melyssa Barata
    • 2
  • Jorge Bernardino
    • 1
    • 2
  • Pedro Furtado
    • 1
  1. 1.CISUC – Centre of Informatics and Systems of the University of CoimbraFCTUC – University of CoimbraCoimbraPortugal
  2. 2.ISEC – Superior Institute of Engineering of CoimbraPolytechnic Institute of CoimbraCoimbraPortugal

Personalised recommendations