Abstract
Clusters of small-scale SMP/CMP nodes are becoming increasingly popular due to their cost-effectiveness. As these nodes are typically capable of supporting a number of network interfaces similar to the number of CPUs, the issue arises how to optimally configure the cluster for optimum communication performance. This paper evaluates a number of configurations on a 4-CPU Opteron cluster with multiple Gigabit Ethernet interfaces. Techniques include channel bonding and using independent communication pathways. With the latter, the use of virtualization via the Xen Virtual Machine Monitor offers the best potential to parallelize all stages of message transmission, for the case when multiple CPUs on a node are communicating simultaneously. Network-level microbenchmarks indicate the best performance is achieved with a configuration where guest virtual machines running on each CPU communicate directly with a dedicated interface, bypassing the virtual machine monitor. Channel bonding also proved to be more effective over multiple communication streams than over single.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aberdeen, D., Baxter, J., Edwards, R.: A 98c/MFLOP Ultra-Large Scale Neural Network Training on a PIII Cluster. In: Proceedings of Supercomputing 2000 (2000)
Capello, F., Richard, O., Etiemble, D.: Understanding performance of SMP clusters running MPI programs. Future Generation Computer Systems 17, 711–720 (2001)
Pukayastha, A., Guiang, C.S., Schulz, K., Minyard, T., Milfeld, K., Barth, W., Hurley, P., Boisseau, J.R.: Performance Characteristics of Dual-processor HPC Cluster Nodes based on 64-bit Commodity Processors. In: Proceedings of the Linux Clusters Institute (LCI) International Conference: the HPC Revolution (2004)
Advanced Microelectronic Devices: AMD Microprocessor Solutions, http://www.amd.com/us-en/Processors
Barham, P., Dragovic, B., Fraser, K., Harris, S.H.T., Ho, A., Neugebauer, R., Pratt, I., Warfield, A.: Xen and the Art of Virtualization. In: Proceedings of SOSP 2003: the Nineteenth ACM Symposium on Operating Systems Principles, pp. 164–177. ACM, New York (2003)
Huang, W., Liu, J., Abali, B., Panda, D.: A Case fopr High Performance Computing with Virtual Machines. In: Proceedings of ICS 2006: International Conference of Supercomputing, Cairns (2006)
Turner, D., Oline, A., Chen, X., Benjegerdes, T.: Integrating New Capabilities into NetPIPE. In: 10th European PVM/MPI User’s Group Meeting, Venice, pp. 37–44. Springer, Heidelberg (2003)
Liu, J., Chandrasekaran, B., Wu, J., Jiang, W., Kini, S., Yu, W., Buntinas, D., Wyckoff, P., Panda, D.: Performance Comparison of MPI Implementations over Infiniband, Myrinet and Quadrics. In: Proceedings of the SuperComputing 2003 Conference, Phoenix (2003)
Menon, A., Jose Renato Santos, A.Y.T., Janakiraman, G., Zwaenepoel, W.: Diagnosing Performance Overheads in the Xen Virtual Machine Environment. In: First ACM/USENIX Conference on Virtual Execution Environments (VEE 2005), pp. 13–25 (2005)
Menon, A., Cox, A.L., Zwaenepoel, W.: Optimizing Network Virtualization in Xen. In: Proceedings of the 2006 USENIX Annual Technical Conference, Boston, pp. 15–28 (2006)
Liu, J., Huang, W., Abali, B., Panda, D.: High Performance VMM-Bypass I/O in Virtual Machines. In: Proceedings of the 2006 USENIX Annual Technical Conference, Boston (2006)
University of Cambridge Computing Laboratory: The Xen virtual machine monitor, http://www.cl.cam.ac.uk/Research/SRG/netos/xen
Willmann, P., Rixner, S., Cox, A.L.: An Evaluation of Network Stack Parallelization Strategies in Modern Operating Systems. Technical Report TR06-872, Rice University Computer Science (2006)
Bhattacharya, S.P., Apte, V.: A Measurement Study of the Linux TCP/IP Stack Performance and Scalability on SMP systems. In: Proceedings of the 1st International Conference on COMmunication Systems softWAre and middlewaRE (COMSWARE), New Delhi (2006)
Nowlabs, Ohio State University: MVAPICH2 Toolset, http://nowlab.cse.ohio-state.edu/projects/mpi-iba/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Strazdins, P., Alexander, R., Barr, D. (2006). Performance Enhancement of SMP Clusters with Multiple Network Interfaces Using Virtualization. In: Min, G., Di Martino, B., Yang, L.T., Guo, M., Rünger, G. (eds) Frontiers of High Performance Computing and Networking – ISPA 2006 Workshops. ISPA 2006. Lecture Notes in Computer Science, vol 4331. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11942634_47
Download citation
DOI: https://doi.org/10.1007/11942634_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-49860-5
Online ISBN: 978-3-540-49862-9
eBook Packages: Computer ScienceComputer Science (R0)