Skip to main content

Advertisement

SpringerLink
Log in
Menu
Find a journal Publish with us
Search
Cart
Book cover

European Conference on Parallel Processing

Euro-Par 2011: Euro-Par 2011: Parallel Processing Workshops pp 398–406Cite as

  1. Home
  2. Euro-Par 2011: Parallel Processing Workshops
  3. Conference paper
A Smart HPC Interconnect for Clusters of Virtual Machines

A Smart HPC Interconnect for Clusters of Virtual Machines

  • Anastassios Nanos30,
  • Nikos Nikoleris31,
  • Stratos Psomadakis30,
  • Elisavet Kozyri32 &
  • …
  • Nectarios Koziris30 
  • Conference paper
  • 1080 Accesses

  • 1 Citations

Part of the Lecture Notes in Computer Science book series (LNTCS,volume 7156)

Abstract

In this paper, we present the design of a VM-aware, high-performance cluster interconnect architecture over 10Gbps Ethernet. Our framework provides a direct data path to the NIC for applications that run on VMs, leaving non-critical paths (such as control) to be handled by intermediate virtualization layers. As a result, we are able to multiplex and prioritize network access per VM. We evaluate our design via a prototype implementation that integrates RDMA semantics into the privileged guest of the Xen virtualization platform. Our framework allows VMs to communicate with the network using a simple user-level RDMA protocol. Preliminary results show that our prototype achieves 681MiB/sec over generic 10GbE hardware and relieves the guest from CPU overheads, while limiting the guest’s CPU utilisation to 34%.

Keywords

  • Virtual Machine
  • Virtual Machine Monitor
  • Ethernet Frame
  • USENIX Annual Technical
  • Grant Mechanism

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Download conference paper PDF

References

  1. Whitaker, A., Shaw, M., Gribble, S.D.: Denali: Lightweight virtual machines for distributed and networked applications. In: Proc. of the USENIX Annual Technical Conference (2002)

    Google Scholar 

  2. PCI SIG: SR-IOV (2007), http://www.pcisig.com/specifications/iov/single_root/

  3. Barham, P., Dragovic, B., Fraser, K., Hand, S., Harris, T., Ho, A., Neugebauer, R., Pratt, I.A., Warfield, A.: Xen and the Art of Virtualization. In: SOSP 2003: Proc. of the 19th ACM Symposium on Operating Systems Principles, pp. 164–177. ACM, NY (2003)

    CrossRef  Google Scholar 

  4. Recio, R., Culley, P., Garcia, D., Hilland, J.: An RDMA Protocol Specification (Version 1.0) This document is a Release Specification of the RDMA Consortium

    Google Scholar 

  5. Goglin, B.: Design and Implementation of Open-MX: High-Performance Message Passing over generic Ethernet hardware. In: CAC 2008: Workshop on Communication Architecture for Clusters, held in conjunction with IPDPS 2008. IEEE Computer Society Press, Miami (2008)

    Google Scholar 

  6. Shalev, L., Satran, J., Borovik, E., Ben-Yehuda, M.: IsoStack—Highly Efficient Network Processing on Dedicated Cores. In: USENIX ATC 2010: USENIX Annual Technical Conference (2010)

    Google Scholar 

  7. Youseff, L., Wolski, R., Gorda, B., Krintz, C.: Evaluating the Performance Impact of Xen on MPI and Process Execution For HPC Systems. In: 1st Intern. Workshop on Virtualization Techn. in Dstrb. Computing. VTDC 2006 (2006)

    Google Scholar 

  8. Nanos, A., Goumas, G., Koziris, N.: Exploring I/O Virtualization Data Paths for MPI Applications in a Cluster of VMs: A Networking Perspective. In: Guarracino, M.R., Vivien, F., Träff, J.L., Cannatoro, M., Danelutto, M., Hast, A., Perla, F., Knüpfer, A., Di Martino, B., Alexander, M. (eds.) Euro-Par-Workshop 2010. LNCS, vol. 6586, pp. 665–671. Springer, Heidelberg (2011)

    CrossRef  Google Scholar 

  9. Menon, A., Cox, A.L., Zwaenepoel, W.: Optimizing network virtualization in Xen. In: ATEC 2006: Proceedings of the Annual Conference on USENIX 2006 Annual Technical Conference, p. 2. USENIX, Berkeley (2006)

    Google Scholar 

  10. Ram, K.K., Santos, J.R., Turner, Y.: Redesigning xen’s memory sharing mechanism for safe and efficient I/O virtualization. In: WIOV 2010: Proceedings of the 2nd Conference on I/O Virtualization, p. 1. USENIX, Berkeley (2010)

    Google Scholar 

  11. Dong, Y., Dai, J., Huang, Z., Guan, H., Tian, K., Jiang, Y.: Towards high-quality I/O virtualization. In: SYSTOR 2009: Proceedings of SYSTOR 2009: The Israeli Experimental Systems Conference, pp. 1–8. ACM, NY (2009)

    CrossRef  Google Scholar 

  12. Santos, J.R., Turner, Y., Janakiraman, G., Pratt, I.: Bridging the gap between software and hardware techniques for I/O virtualization. In: ATC 2008: USENIX 2008 Annual Technical Conference on Annual Technical Conference, pp. 29–42. USENIX, Berkeley (2008)

    Google Scholar 

  13. Ram, K.K., Santos, J.R., Turner, Y., Cox, A.L., Rixner, S.: Achieving 10 Gb/s using safe and transparent network interface virtualization. In: VEE 2009: Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, pp. 61–70. ACM, NY (2009)

    CrossRef  Google Scholar 

  14. Liu, J., Huang, W., Abali, B., Panda, D.K.: High performance VMM-bypass I/O in virtual machines. In: ATEC 2006: Proceedings of the Annual Conference on USENIX 2006 Annual Technical Conference, p. 3. USENIX, Berkeley (2006)

    Google Scholar 

  15. Nanos, A., Koziris, N.: MyriXen: Message Passing in Xen Virtual Machines over Myrinet and Ethernet. In: 4th Workshop on Virtualization in High-Performance Cloud Computing, The Netherlands (2009)

    Google Scholar 

  16. Jones, R.: Netperf, http://www.netperf.org

Download references

Author information

Authors and Affiliations

  1. Computing Systems Laboratory, National Technical University of Athens, Greece

    Anastassios Nanos, Stratos Psomadakis & Nectarios Koziris

  2. Uppsala Architecture Research Team, Uppsala University, Sweden

    Nikos Nikoleris

  3. Cornell University, USA

    Elisavet Kozyri

Authors
  1. Anastassios Nanos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikos Nikoleris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stratos Psomadakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elisavet Kozyri
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nectarios Koziris
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Scilytics, Koellnerhofgasse 3/15A, 1010, Vienna, Austria

    Michael Alexander

  2. ICAR-CNR, Via P. Castellino, 111, 80131, Napoli, Italy

    Pasqua D’Ambra

  3. University of Amsterdam, 1090, Amsterdam, Netherlands

    Adam Belloum

  4. Innovative Computing Laboratory, The University of Tennessee, US

    George Bosilca

  5. Department of Experimental Medicine and Clinic, University Magna Græcia, 88100, Catanzaro, Italy

    Mario Cannataro

  6. Computer Science Department, University of Pisa, Italy

    Marco Danelutto

  7. Second University of Naples, Italy

    Beniamino Di Martino

  8. TUMünchen,, Boltzmannstr. 3, ,, 85748, Garching, Germany

    Michael Gerndt

  9. Equipe Runtime, INRIA Bordeaux Sud-Ouest, 33405, Talence Cedex, France

    Emmanuel Jeannot & Raymond Namyst & 

  10. Equipe HIEPACS, INRIA Bordeaux Sud-Ouest, 33405, Talence Cedex, France

    Jean Roman

  11. Computer Science and Mathematics Division, Oak Ridge National Laboratory, 37831-6164, Oak Ridge, TN, USA

    Stephen L. Scott

  12. Department of Scientific Computing, University of Vienna, Nordbergstr. 15/3C, 1090, Vienna, Austria

    Jesper Larsson Traff

  13. Computer Science and Mathematics Division, Oak Ridge National Laboratory, 37831, Oak Ridge, TN, USA

    Geoffroy Vallée

  14. Technische Universität München, Germany

    Josef Weidendorfer

Rights and permissions

Reprints and Permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nanos, A., Nikoleris, N., Psomadakis, S., Kozyri, E., Koziris, N. (2012). A Smart HPC Interconnect for Clusters of Virtual Machines. In: Alexander, M., et al. Euro-Par 2011: Parallel Processing Workshops. Euro-Par 2011. Lecture Notes in Computer Science, vol 7156. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29740-3_45

Download citation

  • .RIS
  • .ENW
  • .BIB
  • DOI: https://doi.org/10.1007/978-3-642-29740-3_45

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29739-7

  • Online ISBN: 978-3-642-29740-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Search

Navigation

  • Find a journal
  • Publish with us

Discover content

  • Journals A-Z
  • Books A-Z

Publish with us

  • Publish your research
  • Open access publishing

Products and services

  • Our products
  • Librarians
  • Societies
  • Partners and advertisers

Our imprints

  • Springer
  • Nature Portfolio
  • BMC
  • Palgrave Macmillan
  • Apress
  • Your US state privacy rights
  • Accessibility statement
  • Terms and conditions
  • Privacy policy
  • Help and support

167.114.118.210

Not affiliated

Springer Nature

© 2023 Springer Nature