Selected Topics in Performance Evaluation and Benchmarking

Volume 7755 of the series Lecture Notes in Computer Science pp 75-91

Revisiting ETL Benchmarking: The Case for Hybrid Flows

  • Alkis SimitsisAffiliated withHP Labs
  • , Kevin WilkinsonAffiliated withHP Labs

* Final gross prices may vary according to local VAT.

Get Access


Modern business intelligence systems integrate a variety of data sources using multiple data execution engines. A common example is the use of Hadoop to analyze unstructured text and merging the results with relational database queries over a data warehouse. These analytic data flows are generalizations of ETL flows. We refer to multi-engine data flows as hybrid flows. In this paper, we present our benchmark infrastructure for hybrid flows and illustrate its use with an example hybrid flow. We then present a collection of parameters to describe hybrid flows. Such parameters are needed to define and run a hybrid flows benchmark. An inherent difficulty in benchmarking ETL flows is the diversity of operators offered by ETL engines. However, a commonality for all engines is extract and load operations, operations which rely on data and function shipping. We propose that by focusing on these two operations for hybrid flows, it may be feasible to revisit the ETL benchmark effort and thus, enable comparison of flows for modern business intelligence applications. We believe our framework may be a useful step toward an industry standard benchmark for ETL flows.