One-Sided Communication on Clusters with Myrinet
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.Get Access
This paper describes an efficient implementation of one-sided communication on top of the GM low-level message-passing library for clusters with Myrinet. This approach is compatible with shared memory, exploits pipelining, nonblocking communication, and overlapping memory registration with memory copy to maximize the transfer rate. The paper addresses critical design issues faced on the commodity clusters and then describes possible solutions for matching the low-level network protocol with user-level interfaces. The performance implications of the design decisions are presented and discussed in context of a standalone communication benchmark as well as two applications. Finally, the paper offers some indications on what additional features would be desirable in a communication library like GM to better support one-sided communication.
- S. Lumetta, A. Mainwaring and D. Culler, Multi-protocol active messages on a cluster of SMP's, in: Proceedings of Supercomputing '97 (1997).
- H. Tezuka, A. Hori, Y. Ishikawa and M. Sato, PM: An operating system coordinated high performance communication library, in: High Performance Computing and Networking, Lecture Notes in Computer Science, Vol. 1225 (Springer, 1997).
- L. Prylli and B. Tourancheau, BIP: A new protocol designed for high performance networking on Myrinet, in: Workshop PC-NOW, IPPS/SPDP98 (Orlando, USA, 1998).
- S. Parkin, M. Lauria, A. Chien et al., High Performance Virtual Machines (HPVM): Clusters with supercomputing APIs and performance, in: 8th SIAM Conference on Parallel Processing for Scientific Computing (PP97) (1997).
- Myricom, The GM Message Passing System (10/16/1999).
- J. Nieplocha and B. Carpenter, ARMCI: A portable remote memory copy library for distributed array libraries and compiler run-time systems, in: Proc. RTSPP IPPS/SDP'99 (1999).
- K. Parzyszek, J. Nieplocha and R. Kendall, A generalized portable SHMEM library for high performance computing, in: Proceedings Int. Conf. on Parallel and Distributed Computing and Systems, PDCS-2000 (2000).
- R. Bariuso and A. Knies, SHMEM's User's Guide, SN-2516, rev. 2.2 (Cray Research, Inc., Eagan, MN, 1994).
- J. Nieplocha, J. Ju and T.P. Straatsma, A multiprotocol communication support for the global address space programming model on the IBM SP, in: Proc. EuroPar-2000, Lecture Notes in Computer Science, Vol. 1900 (Springer, 2000).
- K. Langendoen, J. Romein, R. Bhoedjang and H. Bal, Integrating polling, interrupts, and thread management, in: Proc. Frontiers'96 (1996).
- J. Hsieh, T. Leng, V. Mashayekhi and R. Rooholamini, Architectural and performance evaluation of GigaNet and Myrinet interconnects on clusters of small-scale SMP servers, in: Proceedings SC2000, http://www.sc2000.org/techpapr/papers/pap.pap294.pdf
- M. Banikazemi, R. Govindaraju, R. Blackmore and D.K. Panda, Implementing efficient MPI on LAPI for the IBM RS/600 SP systems: Experiences and performance evaluation, in: Proc. IPPS99 (1999).
- A. Gallatin, J. Chase and K. Yocum, Trapeze/IP: TCP/IP at near-gigabit speeds, in: 1999 USENIX Technical Conference, June 1999.
- K. Moldeklev and P. Gunningberg, How a large ATM MTU causes deadlocks in TCP data transfers, IEEE/ACMTransactions on Networking 3(4) (August 1995).
- J. Nieplocha, J. Ju and E. Apra, One-sided communication on SMP clusters with Myrinet using the GM message-passing library, in: Proc. CAC'01/IPDPS'01 (2001).
- Myricom, Portable MPI Model Implementation over GM, version 1.2.1 (file README-GM) (23 June, 2000).
- L.A. Giannini and A. Chien, A software architecture for Global address space on communication on clusters: Put/Get on Fast Messages, in: 7th Int. IEEE Symposium on High Performance Distributed Computing, HPDC-7 (1998).
- S.C. Woo, M.O. Ohara, E. Torrie, J.P. Singh and A. Gupta, The SPLASH-2 programs: Characterization and methodological considerations, in: Proc. 22nd International Symposium on Computer Architecture (1995).
- High Performance Computational Chemistry Group, NWChem, A Computational Chemistry Package for Parallel Computers, Version 4.0, Pacific Northwest National Laboratory, Richland, WA (2000), http://www.emsl.pnl.gov:2080/docs/nwchem
- One-Sided Communication on Clusters with Myrinet
Volume 6, Issue 2 , pp 115-124
- Cover Date
- Print ISSN
- Online ISSN
- Kluwer Academic Publishers
- Additional Links
- one-sided communication
- remote memory operations
- Industry Sectors