A Performance Evaluation of Public Cloud Using TPC-C

  • Jinhui Yao
  • Alex Ng
  • Shiping Chen
  • Dongxi Liu
  • Carsten Friedrich
  • Surya Nepal
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7759)


Cloud is becoming the next-generation computing paradigm for enterprises to deploy services and run business. While most Cloud service providers promise some Quality of Service (QoS) through a Service Level Agreement (SLA), it is very hard for Cloud clients to know what impacts these QoS have on their businesses. In this paper, we study this issue by conducting a simple performance evaluation of two public Clouds. We selected TPC-C to benchmark three types of instances (Small, Medium and Large) provided by the Cloud providers in order to find out how the typical online transaction process systems perform on the cloud nodes. Our testing results show that the different Cloud environments deliver very different performance landscapes with different Cloud instances. Our work demonstrates the importance and opportunity to choose the appropriate Cloud instance in achieving an optimal cost-performance ratio for a class of cloud applications.


Cloud Computing Public Cloud QoS Service Level Agreement Performance TPC-C Benchmarking 


  1. 1.
    TPC-C Home Page, (accessed April 26, 2012)
  2. 2.
    Understanding Full Virtualization, Paravirtualization, and Hardware Assist, VMWare (2007)Google Scholar
  3. 3.
    Armstrong, D., Djemame, K.: Towards Quality of Service in the Cloud. In: Proceedings of the 25th UK Performance Engineering Workshop, Leeds, UK (2009)Google Scholar
  4. 4.
    Bautista, L., Abran, A., April, A.: Design of a Performance Measurement Framework for Cloud Computing. Journal of Software Engineering and Applications 5(2), 69–75 (2012)CrossRefGoogle Scholar
  5. 5.
    Dejun, J., Pierre, G., Chi, C.-H.: EC2 Performance Analysis for Resource Provisioning of Service-Oriented Applications. In: Dan, A., Gittler, F., Toumani, F. (eds.) ICSOC/ServiceWave 2009. LNCS, vol. 6275, pp. 197–207. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  6. 6.
    Ferretti, S., Ghini, V., Panzieri, F., et al.: QoS-Aware Clouds. In: Proceedings of the IEEE 3rd International Conference on Cloud Computing (2010)Google Scholar
  7. 7.
    Fito, J.O., Goiri, I., Guitart, J.: SLA-driven Elastic Cloud Hosting Provider. In: Proceedings of the 18th Euromicro Conference on Parallel, Distributed and Network-based Processing (2010)Google Scholar
  8. 8.
    Gupta, D., Cherkasova, L., Gardner, R., Vahdat, A.: Enforcing Performance Isolation Across Virtual Machines in Xen. In: van Steen, M., Henning, M. (eds.) Middleware 2006. LNCS, vol. 4290, pp. 342–362. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Iosup, A., Ostermann, S., Yigitbasi, M.N., et al.: Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing. IEEE Transactions on Parallel and Distributed Systems 22(6), 931–945 (2011)CrossRefGoogle Scholar
  10. 10.
    Li, A., Yang, X., Kandula, S., et al.: CloudCmp: comparing public cloud providers. In: Proceedings of the 10th Annual Conference on Internet Measurement (IMC 2010), Melbourne, Australia, pp. 1–14 (2010)Google Scholar
  11. 11.
    McKendrick, J.: Cloud Will Generate 14 Million Jobs By 2015: That’s A Good Start (March 5, 2012), (accessed April 16, 2012)
  12. 12.
    Patel, P., Ranabahu, A., Sheth, A.: Service Level Agreement in Cloud Computing. In: Proceedings of the Cloud Workshop at OOPSLA (2009)Google Scholar
  13. 13.
    Rochwerger, B., Breitgand, D., Levy, E., et al.: The Reservoir model and architecture for open federated cloud computing. IBM Journal of Research and Development 53(4) (2009)Google Scholar
  14. 14.
    Wang, G., Ng, T.S.E.: The Impact of Virtualization on Network Performance of Amazon EC2 Data Center. In: Proceedings of the INFOCOM 2010 (2010)Google Scholar
  15. 15.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jinhui Yao
    • 1
    • 3
  • Alex Ng
    • 2
  • Shiping Chen
    • 1
    • 3
  • Dongxi Liu
    • 3
  • Carsten Friedrich
    • 3
  • Surya Nepal
    • 3
  1. 1.School of Electrical and Information EngineeringUniversity of SydneyAustralia
  2. 2.The University of BallaratAustralia
  3. 3.Information Engineering LaboratoryCSIRO ICT CentreAustralia

Personalised recommendations