Abstract
A major obstacle to virtualizing HPC workloads is a concern about the performance loss due to virtualization. We will demonstrate that new features significantly enhance the performance and scalability of virtualized HPC workloads on VMware’s virtualization platform. Specifically, we will discuss VMware’s ESXi Server performance for virtual machines with up to 64 virtual CPUs as well as support for exposing virtual NUMA topology to guest operating systems, enabling the operating system and applications to make intelligent NUMA aware decisions about memory allocation and process/thread placement. NUMA support is especially important for large VMs which necessarily span host NUMA nodes on all modern hardware. We will show how the virtual NUMA topology is chosen to closely match physical host topology, while preserving the now expected virtualization benefits of portability and load balancing. We show that the benefit of exposing the virtual NUMA topology can lead to performance gains of up to 167%. Overall, we will show close to native performance on applications from SPEC MPI V2.0 and SPEC OMP V3.2 benchmarks virtualized on our prototype VMware’s ESXi Server.
Chapter PDF
Similar content being viewed by others
Keywords
References
Advanced Configuration and Power Interface specification, rev 4.0a (2009), http://www.acpi.info/spec40.htm
Agesen, O., Garthwaite, A., Sheldon, J., Subrahmanyam, P.: The evolution of an x86 virtual machine monitor. Operating Systems Review 44(4) (2010)
Scalable Infrastructure with the CPU scheduler in VMware ESX 4.1, http://www.vmware.com/files/pdf/techpaper/VMW_vSphere41_cpu_schedule_ESX.pdf
Gavrilovska, A., Kumar, S., et al.: High-Performance Hypervisor Architectures: Virtualization in HPC Systems. In: HPCVirt 2007: 1st Workshop on System-level Virtualization for High Performance Computing (2007)
Huang, W., Gao, Q., Liu, J., Panda, D.K.: High performance virtual machine migration with RDMA over modern interconnects. In: Proceedings of the 2007 IEEE International Conference on Cluster Computing, CLUSTER 2007, pp. 11–20. IEEE Computer Society, Washington, DC (2007)
Mergen, M.F., Uhlig, V., Krieger, O., Xenidis, J.: Virtualization for high-performance computing. SIGOPS Oper. Syst. Rev. 40, 8–11 (2006)
Nagarajan, A.B., Mueller, F.: Proactive fault tolerance for HPC with Xen Virtualization. In: Proceedings of the 21st Annual International Conference on Supercomputing (ICS 2007), pp. 23–32. ACM Press (2007)
Open MPI: Open Source High Performance Computing (2011), http://www.open-mpi.org
Ranadive, A., Kesavan, M., Gavrilovska, A., Schwan, K.: Performance implications of virtualizing multicore cluster machines. In: Proceedings of the 2nd Workshop on System-Level Virtualization for High Performance Computing, HPCVirt 2008, pp. 1–8. ACM, New York (2008)
Rao, D., Schwan, K.: vNUMA-mgr: Managing VM memory on NUMA platforms. In: 2010 International Conference on High Performance Computing (HiPC), pp. 1–10 (December 2010)
Simons, J.E., Buell, J.: Virtualizing high performance computing. SIGOPS Oper. Syst. Rev. 44, 136–145 (2010)
Valle, G., Engelmann, C., Scott, S.L., Naughton, T., Ong, H.: System-Level Virtualization for High Performance Computing, February 13-15 (2008)
Waldspurger, C.A.: Memory resource management in VMware ESX server. In: OSDI 2002: Proceedings of the 5th Symposium on Operating Systems Design and Implementation, pp. 181–194. ACM Press, New York (2002)
Youseff, L., Seymour, K., You, H., Dongarra, J., Wolski, R.: The impact of paravirtualized memory hierarchy on linear algebra computational kernels and software. In: Proceedings of the 17th International Symposium on High Performance Distributed Computing, HPDC 2008, pp. 141–152. ACM, New York (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ali, Q., Kiriansky, V., Simons, J., Zaroo, P. (2012). Performance Evaluation of HPC Benchmarks on VMware’s ESXi Server. In: Alexander, M., et al. Euro-Par 2011: Parallel Processing Workshops. Euro-Par 2011. Lecture Notes in Computer Science, vol 7155. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29737-3_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-29737-3_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-29736-6
Online ISBN: 978-3-642-29737-3
eBook Packages: Computer ScienceComputer Science (R0)