Selected Topics in Performance Evaluation and Benchmarking

Volume 7755 of the series Lecture Notes in Computer Science pp 173-188

Benchmarking in the Cloud: What It Should, Can, and Cannot Be

  • Enno FolkertsAffiliated withSAP AG
  • , Alexander AlexandrovAffiliated withTU Berlin
  • , Kai SachsAffiliated withSAP AG
  • , Alexandru IosupAffiliated withDelft University of Technology
  • , Volker MarklAffiliated withTU Berlin
  • , Cafer TosunAffiliated withSAP AG

* Final gross prices may vary according to local VAT.

Get Access


With the increasing adoption of Cloud Computing, we observe an increasing need for Cloud Benchmarks, in order to assess the performance of Cloud infrastructures and software stacks, to assist with provisioning decisions for Cloud users, and to compare Cloud offerings. We understand our paper as one of the first systematic approaches to the topic of Cloud Benchmarks. Our driving principle is that Cloud Benchmarks must consider end-to-end performance and pricing, taking into account that services are delivered over the Internet. This requirement yields new challenges for benchmarking and requires us to revisit existing benchmarking practices in order to adopt them to the Cloud.