Technology Conference on Performance Evaluation and Benchmarking

TPCTC 2014: Performance Characterization and Benchmarking. Traditional to Big Data pp 1-12

Introducing TPCx-HS: The First Industry Standard for Benchmarking Big Data Systems

  • Raghunath Nambiar
  • Meikel Poess
  • Akon Dey
  • Paul Cao
  • Tariq Magdon-Ismail
  • Da Qi Ren
  • Andrew Bond
Conference paper

DOI: 10.1007/978-3-319-15350-6_1

Volume 8904 of the book series Lecture Notes in Computer Science (LNCS)
Cite this paper as:
Nambiar R. et al. (2015) Introducing TPCx-HS: The First Industry Standard for Benchmarking Big Data Systems. In: Nambiar R., Poess M. (eds) Performance Characterization and Benchmarking. Traditional to Big Data. TPCTC 2014. Lecture Notes in Computer Science, vol 8904. Springer, Cham

Abstract

The designation Big Data has become a mainstream buzz phrase across many industries as well as research circles. Today many companies are making performance claims that are not easily verifiable and comparable in the absence of a neutral industry benchmark. Instead one of the test suites used to compare performance of Hadoop based Big Data systems is the TeraSort. While it nicely defines the data set and tasks to measure Big Data Hadoop systems it lacks a formal specification and enforcement rules that enable the comparison of results across systems. In this paper we introduce TPCx-HS, the industry’s first industry standard benchmark, designed to stress both hardware and software that is based on Apache HDFS API compatible distributions. TPCx-HS extends the workload defined in TeraSort with formal rules for implementation, execution, metric, result verification, publication and pricing. It can be used to asses a broad range of system topologies and implementation methodologies of Big Data Hadoop systems in a technically rigorous and directly comparable and vendor-neutral manner.

Keywords

TPC Big Data Industry standard Benchmark 

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Raghunath Nambiar
    • 1
  • Meikel Poess
    • 2
  • Akon Dey
    • 3
  • Paul Cao
    • 4
  • Tariq Magdon-Ismail
    • 5
  • Da Qi Ren
    • 6
  • Andrew Bond
    • 7
  1. 1.Cisco Systems, Inc.San JoseUSA
  2. 2.Oracle CorporationRedwood ShoresUSA
  3. 3.School of Information TechnologiesUniversity of SydneySydneyAustralia
  4. 4.Hewlett-PackardHoustonUSA
  5. 5.VMware, Inc.Palo AltoUSA
  6. 6.Futurewei TechnologiesSanta ClaraUSA
  7. 7.Red HatRaleighUSA