BDMS Performance Evaluation: Practices, Pitfalls, and Possibilities

  • Michael J. Carey
Conference paper

DOI: 10.1007/978-3-642-36727-4_8

Part of the Lecture Notes in Computer Science book series (LNCS, volume 7755)
Cite this paper as:
Carey M.J. (2013) BDMS Performance Evaluation: Practices, Pitfalls, and Possibilities. In: Nambiar R., Poess M. (eds) Selected Topics in Performance Evaluation and Benchmarking. TPCTC 2012. Lecture Notes in Computer Science, vol 7755. Springer, Berlin, Heidelberg


Much of the IT world today is buzzing about Big Data, and we are witnessing the emergence of a new generation of data-oriented platforms aimed at storing and processing all of the anticipated Big Data. The current generation of Big Data Management Systems (BDMSs) can largely be divided into two kinds of platforms: systems for Big Data analytics, which today tend to be batch-oriented and based on MapReduce (e.g., Hadoop), and systems for Big Data storage and front-end request-serving, which are usually based on key-value (a.k.a. NoSQL) stores. In this paper we ponder the problem of evaluating the performance of such systems. After taking a brief historical look at Big Data management and DBMS benchmarking, we begin our pondering of BDMS performance evaluation by reviewing several key recent efforts to measure and compare the performance of BDMSs. Next we discuss a series of potential pitfalls that such evaluation efforts should watch out for, pitfalls mostly based on the author’s own experiences with past benchmarking efforts. Finally, we close by discussing some of the unmet needs and future possibilities with regard to BDMS performance characterization and assessment efforts.


Data-intensive computing Big Data performance benchmarking MapReduce Hadoop key-value stores NoSQL systems 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Michael J. Carey
    • 1
  1. 1.Information Systems Group, Computer Sciences DepartmentUniversity of California, IrvineIrvineUSA

Personalised recommendations