Performance Characterization and Benchmarking. Traditional to Big Data

Volume 8904 of the series Lecture Notes in Computer Science pp 44-63


Discussion of BigBench: A Proposed Industry Standard Performance Benchmark for Big Data

  • Chaitanya BaruAffiliated withSan Diego Supercomputer Center
  • , Milind BhandarkarAffiliated withPivotal
  • , Carlo CurinoAffiliated withMicrosoft Corporation
  • , Manuel DanischAffiliated withBankmark
  • , Michael FrankAffiliated withBankmark
  • , Bhaskar GowdaAffiliated withIntel Corporation
  • , Hans-Arno JacobsenAffiliated withMiddleware Systems Research Group
  • , Huang JieAffiliated withIntel Corporation
  • , Dileep KumarAffiliated withCloudera
    • , Raghunath NambiarAffiliated withCisco Systems
    • , Meikel PoessAffiliated withOracle Corporation
    • , Francois RaabAffiliated withInfosizing
    • , Tilmann RablAffiliated withBankmarkMiddleware Systems Research Group Email author 
    • , Nishkam RaviAffiliated withCloudera
    • , Kai SachsAffiliated withSPEC Research Group
    • , Saptak SenAffiliated withHortonworks
    • , Lan YiAffiliated withIntel Corporation
    • , Choonhan YounAffiliated withSan Diego Supercomputer Center

* Final gross prices may vary according to local VAT.

Get Access


Enterprises perceive a huge opportunity in mining information that can be found in big data. New storage systems and processing paradigms are allowing for ever larger data sets to be collected and analyzed. The high demand for data analytics and rapid development in technologies has led to a sizable ecosystem of big data processing systems. However, the lack of established, standardized benchmarks makes it difficult for users to choose the appropriate systems that suit their requirements. To address this problem, we have developed the BigBench benchmark specification. BigBench is the first end-to-end big data analytics benchmark suite. In this paper, we present the BigBench benchmark and analyze the workload from technical as well as business point of view. We characterize the queries in the workload along different dimensions, according to their functional characteristics, and also analyze their runtime behavior. Finally, we evaluate the suitability and relevance of the workload from the point of view of enterprise applications, and discuss potential extensions to the proposed specification in order to cover typical big data processing use cases.