Advertisement

Evaluation of the HPC Challenge Benchmarks in Virtualized Environments

  • Piotr Luszczek
  • Eric Meek
  • Shirley Moore
  • Dan Terpstra
  • Vincent M. Weaver
  • Jack Dongarra
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7156)

Abstract

This paper evaluates the performance of the HPC Challenge benchmarks in several virtual environments, including VMware, KVM and VirtualBox. The HPC Challenge benchmarks consist of a suite of tests that examine the performance of HPC architectures using kernels with memory access patterns more challenging than those of the High Performance LINPACK (HPL) benchmark used in the TOP500 list. The tests include four local (matrix-matrix multiply, STREAM, RandomAccess and FFT) and four global (High Performance Linpack – HPL, parallel matrix transpose – PTRANS, RandomAccess and FFT) kernel benchmarks.

The purpose of our experiments is to evaluate the overheads of the different virtual environments and investigate how different aspects of the system are affected by virtualization. We ran the benchmarks on an 8-core system with Core i7 processors using Open MPI. We did runs on the bare hardware and in each of the virtual environments for a range of problem sizes. As expected, the HPL results had some overhead in all the virtual environments, with the overhead becoming less significant with larger problem sizes. The RandomAccess results show drastically different behavior and we attempt to explain it with pertinent experiments. We show the cause of variability of performance results as well as major causes of measurement error.

Keywords

Virtual Machine Virtual Environment Problem Size High Performance Computing Bare Metal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Danciu, V.A., gentschen Felde, N., Kranzlmüller, D., Lindinger, T.: High-performance aspects in virtualized infrastructures. In: 4th International DMTF Academic Alliance Workshop on Systems and Virtualization Management, pp. 25–32 (October 2010)Google Scholar
  2. 2.
    Han, J., Ahn, J., Kim, C., Kwon, Y., Choi, Y.-r., Huh, J.: The Effect of Multi-core on HPC Applications in Virtualized Systems. In: Guarracino, M.R., Vivien, F., Träff, J.L., Cannatoro, M., Danelutto, M., Hast, A., Perla, F., Knüpfer, A., Di Martino, B., Alexander, M. (eds.) Euro-Par-Workshop 2010. LNCS, vol. 6586, pp. 615–623. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  3. 3.
    Hines, S., Wyatt, B., Chang, J.M.: Increasing timing resolution for processes and threads in Linux (2000) (unpublished)Google Scholar
  4. 4.
    Huang, W., Liu, J., Abali, B., Panda, D.: A case for high performance computing with virtual machines. In: Proceedings of the 20th Annual International Conference on Supercomputing (2006)Google Scholar
  5. 5.
    Li, A., Yang, X., Kandula, S., Zhang, M.: CloudCmp: comparing public cloud providers. In: 10th Annual Conference on Internet Measurement (2010)Google Scholar
  6. 6.
    Luszczek, P., Bailey, D., Dongarra, J., Kepner, J., Lucas, R., Rabenseifner, R., Takahashi, D.: The HPC challenge HPCC benchmark suite. In: SuperComputing 2006 Conference Tutorial (2006)Google Scholar
  7. 7.
    Mei, Y., Liu, L., Pu, X., Sivathanu, S.: Performance measurements and analysis of network I/O applications in virtualized cloud. In: IEEE 3rd International Conference on Cloud Computing, pp. 59–66 (August 2010)Google Scholar
  8. 8.
    Smith, K.A., Selzter, M.: File layout and file system performance. Computer Science Technical Report TR-35-94, Harvard University (1994)Google Scholar
  9. 9.
    Tao, J., Fürlinger, K., Marten, H.: Performance Evaluation of OpenMP Applications on Virtualized Multicore Machines. In: Chapman, B.M., Gropp, W.D., Kumaran, K., Müller, M.S. (eds.) IWOMP 2011. LNCS, vol. 6665, pp. 138–150. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  10. 10.
    Tsugawa, M., Fortes, J.A.B.: Characterizing user-level network virtualization: performance, overheads and limits. International Journal of Network Management (2009), doi:10.1002/nem.733Google Scholar
  11. 11.
    Timekeeping in VMware Virtual Machines: VMware ESX 4.0/ESXi 4.0, VMware workstation 7.0 information guideGoogle Scholar
  12. 12.
    Walters, J., Chaudhary, V., Cha, M., Guercio, S.J., Gallo, S.: A comparison of virtualization technologies for HPC. In: 22nd International Conference on Advanced Information Networking and Applications, pp. 861–868 (March 2008)Google Scholar
  13. 13.
    Younge, A.J., Henschel, R., Brown, J.T., von Laszewski, G., Qiu, J., Fox, G.C.: Analysis of virtualization technologies for High Performance Computing environments. In: Proceedings of The Fourth IEEE International Conference on Cloud Computing (CLOUD 2011), Washington Marriott, Washington DC, USA, July 4-9 (2011); technical Report (February 15, 2011), updated (April 2011)Google Scholar
  14. 14.
    Youseff, L., Wolski, R., Gorda, B., Krintz, C.: Paravirtualization for HPC Systems. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds.) ISPA Workshops 2006. LNCS, vol. 4331, pp. 474–486. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Piotr Luszczek
    • 1
  • Eric Meek
    • 1
  • Shirley Moore
    • 1
  • Dan Terpstra
    • 1
  • Vincent M. Weaver
    • 1
  • Jack Dongarra
    • 1
  1. 1.Innovative Computing LaboratoryUniversity of TennesseeKnoxvilleUSA

Personalised recommendations