Encyclopedia of Big Data Technologies

2019 Edition
| Editors: Sherif Sakr, Albert Y. Zomaya

End-to-End Benchmark

  • Milind A. BhandarkarEmail author
Reference work entry
DOI: https://doi.org/10.1007/978-3-319-77525-8_112



An end-to-end data pipeline benchmark is a standardized suite of data ingestion, data processing, and data queries, arranged in a series of stages, where the output of a previous stage in a pipeline feeds the next stage in pipeline, exercising all the needed system characteristics for commonly constructed data pipeline workloads.

Historical Background

As we witness the rapid transformation in data architecture, where relational database management systems (RDBMS) are being supplemented by large-scale non-relational stores such as Hadoop Distributed File System (HDFS), MongoDB, Apache Cassandra, and Apache HBase, a more fundamental shift is on its way, which would require larger changes to modern data architectures. While the current shift was mandated by business requirements for the connected world, the next wave will be dictated by operational cost optimization, transformative changes in the underlying infrastructure...

This is a preview of subscription content, log in to check access.


  1. Baru C, Bhandarkar M, Nambiar R, Poess M, Rabl T (2013) Benchmarking big data systems and the big data top100 list. Big Data 1(1):60–64CrossRefGoogle Scholar
  2. Ghazal A, Rabl T, Hu M, Raab F, Poess M, Crolotte A, Jacobsen HA (2013) Bigbench: towards an industry standard benchmark for big data analytics. In Proceedings of the 2013 ACM SIGMOD international conference on management of data, SIGMOD’13, ACM, New York, pp 1197–1208Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2019

Authors and Affiliations

  1. 1.AmpoolInc., Santa ClaraCAUSA

Section editors and affiliations

  • Meikel Poess
    • 1
  • Tilmann Rabl
    • 2
  1. 1.Server TechnologiesOracleRedwood ShoresUSA
  2. 2.Database Systems and Information Management GroupTechnische Universität BerlinBerlinGermany