Evaluating Cloud Storage Services for Tightly-Coupled Applications

  • Alexandra Carpen-Amarie
  • Kate Keahey
  • John Bresnahan
  • Gabriel Antoniu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7640)


The emergence of Cloud computing has given rise to numerous attempts to study the portability of scientific applications to this new paradigm. Tightly-coupled applications are a common class of scientific HPC applications, which exhibit specific requirements previously addressed by supercomputers. A key challenge towards the adoption of the Cloud paradigm for such applications is data management. In this paper, we argue that Cloud storage services represent a suitable data storage and sharing option for Cloud applications. We evaluate a distributed storage plugin for Cumulus, an S3-compatible open-source Cloud service, and we conduct a series of experiments with an atmospheric modeling application running in a private Cloud deployed on the Grid’5000 testbed. Our results, obtained on up to 144 parallel processes, show that the application is able to scale with the size of the data and the number of processes, while storing 50 GB of output data on a Cloud storage service.


Cloud computing data management Cloud storage service HPC applications Nimbus Cumulus 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amazon Elastic Block Store (EBS), http://aws.amazon.com/ebs/
  2. 2.
    Amazon Elastic Compute Cloud (EC2), http://aws.amazon.com/ec2/
  3. 3.
    Bresnahan, J., Keahey, K., LaBissoniere, D., et al.: Cumulus: an open source storage cloud for science. In: ScienceCloud 2011, pp. 25–32. ACM, New York (2011)Google Scholar
  4. 4.
    Bryan, G.H., Rotunno, R.: Evaluation of an analytical model for the maximum intensity of tropical cyclones. Journal of the Atmospheric Sciences 66(10), 3042–3060 (2009)CrossRefGoogle Scholar
  5. 5.
    Carlyle, A.G., Harrell, S.L., Smith, P.M.: Cost-effective HPC: The community or the Cloud? In: CloudCom 2010, pp. 169–176 (December 2010)Google Scholar
  6. 6.
  7. 7.
    Ekanayake, J., Fox, G.: High Performance Parallel Computing with Clouds and Cloud Technologies. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) Cloudcomp 2009. LNICST, vol. 34, pp. 20–38. Springer, Heidelberg (2010)Google Scholar
  8. 8.
    El-Khamra, Y., Hyunjoo, K., Shantenu, J., et al.: Exploring the performance fluctuations of HPC workloads on clouds. In: CloudCom, pp. 383–387 (December 2010)Google Scholar
  9. 9.
    File System in UserspacE (FUSE), http://fuse.sourceforge.net
  10. 10.
    Gropp, W., Lusk, E., et al.: High-performance, portable implementation of the MPI Message Passing Interface Standard. Parallel Computing 22(6), 789–828 (1996)MATHCrossRefGoogle Scholar
  11. 11.
    He, Q., Zhou, S., Kobler, B., et al.: Case study for running HPC applications in public clouds. In: HPDC 2010, USA, pp. 395–401 (2010)Google Scholar
  12. 12.
    Jégou, Y., Lantéri, S., Leduc, J., et al.: Grid’5000: a large scale and highly reconfigurable experimental grid testbed. Intl. J. of HPC Applications 20(4), 481–494 (2006)Google Scholar
  13. 13.
    Keahey, K., Freeman, T.: Science Clouds: Early Experiences in Cloud Computing for Scientific Applications. In: Cloud Computing and Its Applications (CCA), USA (2008)Google Scholar
  14. 14.
    Ligon, W.B., Ross, R.B.: Implementation and performance of a parallel file system for high performance distributed applications. In: HPDC 1996, pp. 471–480. IEEE Computer Society, Washington, DC (1996)Google Scholar
  15. 15.
    Nicolae, B., Antoniu, G., Bougé, L., et al.: BlobSeer: Next generation data management for large scale infrastructures. J. Parallel and Distrib. Comput. 71(2), 168–184 (2011)Google Scholar
  16. 16.
    Ostermann, S., Iosup, A., Yigitbasi, N., Prodan, R., Fahringer, T., Epema, D.: A Performance Analysis of EC2 Cloud Computing Services for Scientific Computing. In: Avresky, D.R., Diaz, M., Bode, A., Ciciani, B., Dekel, E. (eds.) Cloudcomp 2009. LNICST, vol. 34, pp. 115–131. Springer, Heidelberg (2010)Google Scholar
  17. 17.
    Amazon Simple Storage Service (S3), http://aws.amazon.com/s3/
  18. 18.
    Schmuck, F.B., Haskin, R.L.: GPFS: A shared-disk file system for large computing clusters. In: Conf. on File and Storage Technologies, FAST, pp. 231–244. USENIX (2002)Google Scholar
  19. 19.
  20. 20.
    Younge, A.J., Henschel, R., Brown, J.T., et al.: Analysis of virtualization technologies for high performance computing environments. In: CLOUD, pp. 9–16 (July 2011)Google Scholar
  21. 21.
    Zhai, Y., Liu, M., Zhai, J., et al.: Cloud versus in-house cluster: Evaluating amazon cluster compute instances for running mpi applications. In: 2011 Intl. Conf. for HPC, Networking, Storage and Analysis, SC, pp. 1–10 (November 2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Alexandra Carpen-Amarie
    • 1
  • Kate Keahey
    • 2
  • John Bresnahan
    • 2
  • Gabriel Antoniu
    • 1
  1. 1.INRIA Rennes - Bretagne Atlantique / IRISAFrance
  2. 2.Argonne National LaboratoryUSA

Personalised recommendations