A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing

  • Simon Ostermann
  • Alexandria Iosup
  • Nezih Yigitbasi
  • Radu Prodan
  • Thomas Fahringer
  • Dick Epema
Part of the Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering book series (LNICST, volume 34)

Abstract

Cloud Computing is emerging today as a commercial infrastructure that eliminates the need for maintaining expensive computing hardware. Through the use of virtualization, clouds promise to address with the same shared set of physical resources a large user base with different needs. Thus, clouds promise to be for scientists an alternative to clusters, grids, and supercomputers. However, virtualization may induce significant performance penalties for the demanding scientific computing workloads. In this work we present an evaluation of the usefulness of the current cloud computing services for scientific computing. We analyze the performance of the Amazon EC2 platform using micro-benchmarks and kernels. While clouds are still changing, our results indicate that the current cloud services need an order of magnitude in performance improvement to be useful to the scientific community.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    The Cloud Status Team. JSON report crawl (January 2009), http://www.cloudstatus.com/
  2. 2.
    The HPCC Team. HPCChallenge results (Sept. 2009), http://icl.cs.utk.edu/hpcc/hpcc_results.cgi
  3. 3.
    Advanced Clustering Tech. Linpack problem size analyzer (December 2008), http://www.advancedclustering.com/
  4. 4.
    Amazon Inc. Amazon Elastic Compute Cloud (Amazon EC2) (September 2009), http://aws.amazon.com/ec2/
  5. 5.
    Arpaci-Dusseau, R.H., Arpaci-Dusseau, A.C., Vahdat, A., Liu, L.T., Anderson, T.E., Patterson, D.A.: The interaction of parallel and sequential workloads on a network of workstations. In: SIGMETRICS, pp. 267-278 (1995)Google Scholar
  6. 6.
    Babcock, M.: XEN benchmarks. Tech. Rep. (August 2007), http://mikebabcock.ca/linux/xen/
  7. 7.
    Barham, P., Dragovic, B., Eraser, K., Hand, S., Harris, T.L., Ho, A., Pratt, I., Warfield, A.: Xen and the art of virtualization. In: SOSP. ACM, New York (2003)Google Scholar
  8. 8.
    Bradshaw, R., Desai, N., Freeman, T., Keahey, K.: A scalable approach to deploying and managing appliances. In: TeraGrid Conference 2007 (June 2007)Google Scholar
  9. 9.
    Bray, T.: Bonnie, 1996 (December 2008), http://www.textuality.com/bonnie/
  10. 10.
    Deelman, E., Singh, G., Livny, M., Berriman, J.B., Good, J.: The cost of doing science on the cloud: the Montage example. In: SC, p. 50. IEEE/ACM (2008)Google Scholar
  11. 11.
    Dongarra, J., et al.: Basic linear algebra subprograms technical forum standard. Int’l. J. of High Perf. App. and Supercomputing 16(1), 1–111 (2002)CrossRefGoogle Scholar
  12. 12.
    Foster, LT., Freeman, T., Keahey, K., Scheftner, D., Sotomayor, B., Zhang, X.: Virtual clusters for grid communities. In: CCGrid, pp. 513–520. IEEE, Los Alamitos (2006)Google Scholar
  13. 13.
    GoGrid. GoGrid cloud-server hosting (September 2009), http://www.gogrid.com
  14. 14.
    Kowalski, A.: Bonnie — file system benchmarks. Tech. Rep., Jefferson Lab (October 2002), http://cc.jlab.org/docs/scicomp/benchmark/bonnie.html
  15. 15.
    Luszczek, P., Bailey, D.H., Dongarra, J., Kepner, J., Lucas, R.F., Rabenseifner, R., Takahashi, D.: S12 — The HPC Challenge (HPCC) benchmark suite. In: SC, p. 213. ACM, New York (2006)Google Scholar
  16. 16.
    Assuncao, A.C.M., Buyya, R.: Evaluating the cost-benefit of using cloud computing to extend the capacity of clusters. In: Kranzlmüller, D., Bode, A., Hegering, H.-G., Casanova, H., Gerndt, M. (eds.) 11th IEEE International Conference on High Performance Computing and Communications, HPCC 2009. ACM, New York (2009)Google Scholar
  17. 17.
    McVoy, L., Staelin, C: LMbench — tools for performance analysis (December 2008), http://www.bitmover.com/lmbench/
  18. 18.
    Mucci, P.J., London, K.S.: Low level architectural characterization benchmarks for parallel computers. Technical Report UT-CS-98-394, U. Tennessee (1998)Google Scholar
  19. 19.
    Nagarajan, A.B., Mueller, F., Engelmann, C., Scott, S.L.: Proactive fault tolerance for HPC with Xen virtualization. In: ICS, pp. 23–32. ACM, New York (2007)Google Scholar
  20. 20.
    Nishimura, H., Maruyama, N., Matsuoka, S.: Virtual clusters on the fly — fast, scalable, and flexible installation. In: CCGrid, pp. 549–556. IEEE, Los Alamitos (2007)Google Scholar
  21. 21.
    Nurmi, D., Wolski, R., Grzegorczyk, C., Obertelli, G., Soman, S., Youseff, L., Zagorodnov, D.: The Eucalyptus open-source cloud-computing system. UCSD Tech. Rep. 2008-10 (2008), http://eucalyptus.cs.ucsb.edu/
  22. 22.
    Ostermann, S., Prodan, R., Fahringer, T.: Extended grids with cloud resource management for scientific computing. In: Grid 2009: IEEE/ACM International Conference on Grid Computing (October 2009)Google Scholar
  23. 23.
    Palankar, M.R., Iamnitchi, A., Ripeanu, M., Garfinkel, S.: Amazon S3 for science grids: a viable solution? In: DADC 2008: Proceedings of the 2008 international workshop on Data-aware distributed computing, pp. 55–64. ACM, New York (2008)CrossRefGoogle Scholar
  24. 24.
    Penguin Computing. Reliable hpc linux systems (September 2009), http://www.penguincomputing.com/
  25. 25.
    Prodan, R., Ostermann, S.: A survey and taxonomy of infrastructure as a service and web hosting cloud providers. In: Grid 2009: IEEE/ACM International Conference on Grid Computing (October 2009)Google Scholar
  26. 26.
    Quétier, B., Néri, V., Cappello, F.: Scalability comparison of four host visualization tools. J. Grid Comput. 5(1), 83–98 (2007)CrossRefGoogle Scholar
  27. 27.
    Sotomayor, N., Keahey, K., Foster, I.: Overhead matters: A model for virtual resource management. In: VTDC, pp. 4–11. IEEE, Los Alamitos (2006)Google Scholar
  28. 28.
    Thain, D., Tannenbaum, T., Livny, M.: Distributed computing in practice: the Condor experience. Conc. & Comp.: Pract. & Exp. 17(2-4), 323–356 (2005)CrossRefGoogle Scholar
  29. 29.
    Walker, E.: Benchmarking Amazon EC2 for HP Scientific Computing. Login 33(5), 18–23 (2008)Google Scholar
  30. 30.
    Wang, P., Turner, G.W., Lauer, D.A., Allen, M., Simms, S., Hart, D., Papakhian, M., Stewart, C.A.: Linpack performance on a geographically distributed linux cluster. In: IPDPS. IEEE, Los Alamitos (2004)Google Scholar
  31. 31.
    Worringen, J., Scholtyssik, K.: MP-MPICH: User documentation & technical notes (June 2002)Google Scholar
  32. 32.
    Youseff, L., Seymour, K., You, H., Dongarra, J., Wolski, R.: The impact of paravirtualized memory hierarchy on linear algebra computational kernels and software. In: HPDC, pp. 141–152. ACM, New York (2008)CrossRefGoogle Scholar
  33. 33.
    Youseff, L., Wolski, R., Gorda, B.C., Krintz, C: Paravirtualization for HPC systems. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds.) ISPA Workshops 2006. LNCS, vol. 4331, pp. 474–486. Springer, Heidelberg (2006)Google Scholar

Copyright information

© ICST Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering 2010

Authors and Affiliations

  • Simon Ostermann
    • 1
  • Alexandria Iosup
    • 2
  • Nezih Yigitbasi
    • 2
  • Radu Prodan
    • 1
  • Thomas Fahringer
    • 1
  • Dick Epema
    • 2
  1. 1.University of InnsbruckAustria
  2. 2.Delft University of TechnologyThe Netherlands

Personalised recommendations