Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

European Conference on Parallel Processing

Euro-Par 2011: Euro-Par 2011: Parallel Processing Workshops pp 436–445Cite as

  1. Home
  2. Euro-Par 2011: Parallel Processing Workshops
  3. Conference paper
Evaluation of the HPC Challenge Benchmarks in Virtualized Environments

Evaluation of the HPC Challenge Benchmarks in Virtualized Environments

  • Piotr Luszczek30,
  • Eric Meek30,
  • Shirley Moore30,
  • Dan Terpstra30,
  • Vincent M. Weaver30 &
  • …
  • Jack Dongarra30 
  • Conference paper
  • 1205 Accesses

  • 10 Citations

  • 3 Altmetric

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7156)

Abstract

This paper evaluates the performance of the HPC Challenge benchmarks in several virtual environments, including VMware, KVM and VirtualBox. The HPC Challenge benchmarks consist of a suite of tests that examine the performance of HPC architectures using kernels with memory access patterns more challenging than those of the High Performance LINPACK (HPL) benchmark used in the TOP500 list. The tests include four local (matrix-matrix multiply, STREAM, RandomAccess and FFT) and four global (High Performance Linpack – HPL, parallel matrix transpose – PTRANS, RandomAccess and FFT) kernel benchmarks.

The purpose of our experiments is to evaluate the overheads of the different virtual environments and investigate how different aspects of the system are affected by virtualization. We ran the benchmarks on an 8-core system with Core i7 processors using Open MPI. We did runs on the bare hardware and in each of the virtual environments for a range of problem sizes. As expected, the HPL results had some overhead in all the virtual environments, with the overhead becoming less significant with larger problem sizes. The RandomAccess results show drastically different behavior and we attempt to explain it with pertinent experiments. We show the cause of variability of performance results as well as major causes of measurement error.

Keywords

  • Virtual Machine
  • Virtual Environment
  • Problem Size
  • High Performance Computing
  • Bare Metal

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This material is based upon work supported in part by the National Science Foundation under Grant No. 0910812 to Indiana University for “FutureGrid: An Experimental, High-Performance Grid Test-bed.” Partners in the FutureGrid project include U. Chicago, U. Florida, San Diego Supercomputer Center - UC San Diego, U. Southern California, U. Texas at Austin, U. Tennessee at Knoxville, U. of Virginia, Purdue I., and T-U. Dresden.

Download conference paper PDF

References

  1. Danciu, V.A., gentschen Felde, N., Kranzlmüller, D., Lindinger, T.: High-performance aspects in virtualized infrastructures. In: 4th International DMTF Academic Alliance Workshop on Systems and Virtualization Management, pp. 25–32 (October 2010)

    Google Scholar 

  2. Han, J., Ahn, J., Kim, C., Kwon, Y., Choi, Y.-r., Huh, J.: The Effect of Multi-core on HPC Applications in Virtualized Systems. In: Guarracino, M.R., Vivien, F., Träff, J.L., Cannatoro, M., Danelutto, M., Hast, A., Perla, F., Knüpfer, A., Di Martino, B., Alexander, M. (eds.) Euro-Par-Workshop 2010. LNCS, vol. 6586, pp. 615–623. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  3. Hines, S., Wyatt, B., Chang, J.M.: Increasing timing resolution for processes and threads in Linux (2000) (unpublished)

    Google Scholar 

  4. Huang, W., Liu, J., Abali, B., Panda, D.: A case for high performance computing with virtual machines. In: Proceedings of the 20th Annual International Conference on Supercomputing (2006)

    Google Scholar 

  5. Li, A., Yang, X., Kandula, S., Zhang, M.: CloudCmp: comparing public cloud providers. In: 10th Annual Conference on Internet Measurement (2010)

    Google Scholar 

  6. Luszczek, P., Bailey, D., Dongarra, J., Kepner, J., Lucas, R., Rabenseifner, R., Takahashi, D.: The HPC challenge HPCC benchmark suite. In: SuperComputing 2006 Conference Tutorial (2006)

    Google Scholar 

  7. Mei, Y., Liu, L., Pu, X., Sivathanu, S.: Performance measurements and analysis of network I/O applications in virtualized cloud. In: IEEE 3rd International Conference on Cloud Computing, pp. 59–66 (August 2010)

    Google Scholar 

  8. Smith, K.A., Selzter, M.: File layout and file system performance. Computer Science Technical Report TR-35-94, Harvard University (1994)

    Google Scholar 

  9. Tao, J., Fürlinger, K., Marten, H.: Performance Evaluation of OpenMP Applications on Virtualized Multicore Machines. In: Chapman, B.M., Gropp, W.D., Kumaran, K., Müller, M.S. (eds.) IWOMP 2011. LNCS, vol. 6665, pp. 138–150. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  10. Tsugawa, M., Fortes, J.A.B.: Characterizing user-level network virtualization: performance, overheads and limits. International Journal of Network Management (2009), doi:10.1002/nem.733

    Google Scholar 

  11. Timekeeping in VMware Virtual Machines: VMware ESX 4.0/ESXi 4.0, VMware workstation 7.0 information guide

    Google Scholar 

  12. Walters, J., Chaudhary, V., Cha, M., Guercio, S.J., Gallo, S.: A comparison of virtualization technologies for HPC. In: 22nd International Conference on Advanced Information Networking and Applications, pp. 861–868 (March 2008)

    Google Scholar 

  13. Younge, A.J., Henschel, R., Brown, J.T., von Laszewski, G., Qiu, J., Fox, G.C.: Analysis of virtualization technologies for High Performance Computing environments. In: Proceedings of The Fourth IEEE International Conference on Cloud Computing (CLOUD 2011), Washington Marriott, Washington DC, USA, July 4-9 (2011); technical Report (February 15, 2011), updated (April 2011)

    Google Scholar 

  14. Youseff, L., Wolski, R., Gorda, B., Krintz, C.: Paravirtualization for HPC Systems. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds.) ISPA Workshops 2006. LNCS, vol. 4331, pp. 474–486. Springer, Heidelberg (2006)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Innovative Computing Laboratory, University of Tennessee, Knoxville, USA

    Piotr Luszczek, Eric Meek, Shirley Moore, Dan Terpstra, Vincent M. Weaver & Jack Dongarra

Authors
  1. Piotr Luszczek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eric Meek
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shirley Moore
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dan Terpstra
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vincent M. Weaver
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jack Dongarra
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Scilytics, Koellnerhofgasse 3/15A, 1010, Vienna, Austria

    Michael Alexander

  2. ICAR-CNR, Via P. Castellino, 111, 80131, Napoli, Italy

    Pasqua D’Ambra

  3. University of Amsterdam, 1090, Amsterdam, Netherlands

    Adam Belloum

  4. Innovative Computing Laboratory, The University of Tennessee, US

    George Bosilca

  5. Department of Experimental Medicine and Clinic, University Magna Græcia, 88100, Catanzaro, Italy

    Mario Cannataro

  6. Computer Science Department, University of Pisa, Italy

    Marco Danelutto

  7. Second University of Naples, Italy

    Beniamino Di Martino

  8. TUMünchen,, Boltzmannstr. 3, ,, 85748, Garching, Germany

    Michael Gerndt

  9. Equipe Runtime, INRIA Bordeaux Sud-Ouest, 33405, Talence Cedex, France

    Emmanuel Jeannot & Raymond Namyst & 

  10. Equipe HIEPACS, INRIA Bordeaux Sud-Ouest, 33405, Talence Cedex, France

    Jean Roman

  11. Computer Science and Mathematics Division, Oak Ridge National Laboratory, 37831-6164, Oak Ridge, TN, USA

    Stephen L. Scott

  12. Department of Scientific Computing, University of Vienna, Nordbergstr. 15/3C, 1090, Vienna, Austria

    Jesper Larsson Traff

  13. Computer Science and Mathematics Division, Oak Ridge National Laboratory, 37831, Oak Ridge, TN, USA

    Geoffroy Vallée

  14. Technische Universität München, Germany

    Josef Weidendorfer

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Luszczek, P., Meek, E., Moore, S., Terpstra, D., Weaver, V.M., Dongarra, J. (2012). Evaluation of the HPC Challenge Benchmarks in Virtualized Environments. In: Alexander, M., et al. Euro-Par 2011: Parallel Processing Workshops. Euro-Par 2011. Lecture Notes in Computer Science, vol 7156. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29740-3_49

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-29740-3_49

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29739-7

  • Online ISBN: 978-3-642-29740-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature