Viability of Virtual Machines in HPC

A State of the Art Analysis
  • Jens BreitbartEmail author
  • Simon Pickartz
  • Josef Weidendorfer
  • Antonello Monti
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10104)


Virtualization is common in various areas ranging from mobiles to large data centers operated by cloud providers. Theoretically, virtualization provides various benefits that could be useful to HPC as well, e.g., suspend a large application before system maintenance or migrate a process before a node fails due to hardware malfunctioning.

In this paper, we analyze the current state of the art of virtual machines for HPC with respect to their performance and energy consumption. Furthermore, we report on our findings on the compatibility of the current HPC software stack with virtual machines and how they complicate application analysis and application tuning, as well as how current HPC hardware limits some benefits of VMs.


Virtual Machine Cloud Provider Memory Bandwidth Page Table Virtualization Layer 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Intel Virtualization Technology for Directed I/O. Technical report, Intel Corporation (2014)Google Scholar
  2. 2.
    Bhargava, R., Serebrin, B., et al.: Accelerating two-dimensional page walks for virtualized systems. In: ASPLOS XIII: Proceedings of the 13th International Conference Architectural Support for Programming Languages and Operating System ACM (2008)Google Scholar
  3. 3.
    de Blanche, A., Lundqvist, T.: Addressing characterization methods for memory contention aware co-scheduling. J. Supercomput. 71(4), 1451–1483 (2015)CrossRefGoogle Scholar
  4. 4.
    Breitbart, J., Trinitis, C., et al.: Case study on co-scheduling for HPC applications. In: Proceedings of International Workshop Scheduling and Resource Management for Parallel and Distributed Systems, September 2015Google Scholar
  5. 5.
    Darling, A., Carey, L., et al.: The Design, Implementation, and Evaluation of mpiBLAST. In: Proceedings of ClusterWorld, June 2003Google Scholar
  6. 6.
    Eklov, D., Nikoleris, N., et al.: Bandwidth bandit: quantitative characterization of memory contention. In: IEEE/ACM International Symposium on Code Generation and Optimization (CGO) 2013, February 2013Google Scholar
  7. 7.
    Huang, W., Gao, Q., et al.: High performance virtual machine migration with RDMA over modern interconnects. In: 2007 IEEE International Conference on Cluster Computing (CLUSTER). IEEE (2007)Google Scholar
  8. 8.
    Huang, W., Liu, J., et al.: Nomad: migrating OS-bypass networks in virtual machines. In: VEE 2007: Proceedings of the 3rd International Conference on Virtual Execution Environments. ACM (2007)Google Scholar
  9. 9.
    Ibrahim, K.Z., Hofmeyr, S.A., et al.: Characterizing the performance of parallel applications on multi-socket virtual machines. In: CCGRID (2011)Google Scholar
  10. 10.
    Kraus, J., Förster, M., et al.: Using LAMA for efficient AMG on hybrid clusters. Comput. Sci. Res. Dev. 28(2), 211–220 (2013)CrossRefGoogle Scholar
  11. 11.
    Lin, H., Balaji, P., et al.: Massively parallel genomic sequence search on the Blue Gene/P architecture. In: International Conference for High Performance Computing, Networking, Storage and Analysis (SC 2008). IEEE (2008)Google Scholar
  12. 12.
    Luszczek, P., Meek, E., Moore, S., Terpstra, D., Weaver, V.M., Dongarra, J.: Evaluation of the HPC challenge benchmarks in virtualized environments. In: Alexander, M., et al. (eds.) Euro-Par 2011. LNCS, vol. 7156, pp. 436–445. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-29740-3_49 CrossRefGoogle Scholar
  13. 13.
    Mars, J., Vachharajani, N., et al.: Contention aware execution: online contention detection and response. In: Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization (CGO) (2010)Google Scholar
  14. 14.
    McCalpin, J.D.: Memory bandwidth and machine balance in current high performance computers. IEEE Computer Society Technical Committee on Computer Architecture (TCCA) Newsletter, December 1995.
  15. 15.
    Pickartz, S., Breitbart, J., et al.: Implications of process-migration in virtualized environments. In: Proceedings of 1th Workshop Co-Scheduling of HPC Applications, January 2016Google Scholar
  16. 16.
    Pickartz, S., Clauss, C., et al.: Application migration in HPC–a driver of the exascale era? In: International Conference High Performance Computing and Simulation (HPCS), July 2016Google Scholar
  17. 17.
    Pickartz, S., Clauss, C., et al.: Non-intrusive migration of MPI processes in OS-bypass networks. In: Parallel and Distributed Processing Symposium Workshops (IPDPSW) (2016)Google Scholar
  18. 18.
    Pickartz, S., Gad, R., Lankes, S., Nagel, L., Süß, T., Brinkmann, A., Krempel, S.: Migration techniques in HPC environments. In: Lopes, L., et al. (eds.) Euro-Par 2014. LNCS, vol. 8806, pp. 486–497. Springer, Cham (2014). doi: 10.1007/978-3-319-14313-2_41 Google Scholar
  19. 19.
    Regola, N., Ducom, J.C.: Recommendations for virtualization technologies in high performance computing. In: CloudCom (2010)Google Scholar
  20. 20.
    Uhlig, R., Neiger, G., et al.: Intel virtualization technology. Computer 38(5), 48–56 (2005)CrossRefGoogle Scholar
  21. 21.
    Younge, A.J., Henschel, R., et al.: Analysis of virtualization technologies for high performance computing environments. In: IEEE International Conference on Cloud Computing (CLOUD). IEEE (2011)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Jens Breitbart
    • 1
    Email author
  • Simon Pickartz
    • 2
  • Josef Weidendorfer
    • 1
  • Antonello Monti
    • 2
  1. 1.Chair for Computer Architecture, Department of InformaticsTechnical University MunichMunichGermany
  2. 2.Institute for Automation of Complex Power Systems, E.ON ERCRWTH Aachen UniversityAachenGermany

Personalised recommendations