Cluster Computing

, Volume 10, Issue 4, pp 409–424 | Cite as

A middleware approach for pipelining communications in clusters

Article
First Online:
Received:
Accepted:

Abstract

The Pipelining Communications Middleware (PCM) approach provides a flexible, simple, high-performance mechanism to connect parallel programs running on high performance computers or clusters. This approach enables parallel programs to communicate and coordinate with each other to address larger problems than a single program can solve. The motivation behind the PCM approach grew out of using files as an intermediate transfer stage between processing by different programs. Our approach supersedes this practice by using streaming data set transfers as an “online” communication channel between simultaneously active parallel programs. Thus, the PCM approach addresses the issue of sending data from a parallel program to another parallel program without exposing details such as number of nodes allocated to the program, specific node identifiers, etc. This paper outlines and analyzes our proposed computation and communication model to provide efficient and convenient communications between parallel programs running on high performance computing systems or clusters. We also discuss the PCM challenges as well as current PCM implementations. Our approach achieves scalability, transparency, coordination, synchronization and flow control, and efficient programming. We experimented with data parallel applications to evaluate the performance of the PCM approach. Our experiment results show that the PCM approach achieves nearly ideal throughput that scales linearly with the underlying network medium speed. PCM performs well with small and large data transfers. Furthermore, our experiments show that network infrastructure plays the most significant role in the PCM performance.

Keywords

Cluster communications Pipelining MPI Middleware 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Beckman, P.H., Fasel, P.K., Humphrey, W.F., Mniszewski, S.M.: Efficient coupling of parallel applications using PAWS. In: The 7th IEEE International Symposium on High Performance Distributed Computing, July 1998 Google Scholar
  2. 2.
    Berman, F., Wolski, R., Casanova, H., Cirne, W., Dail, H., Faerman, M., Figueira, S., Hayes, J., Obertelli, G., Schopf, J., Shao, G., Smallen, S., Spring, N., Su, A., Zagorodnov, D.: MPICH-G2: a grid-enabled implementation of the message passing interface. IEEE Trans. Parallel Distributed Syst. 14(4), 369–382 (2003) CrossRefGoogle Scholar
  3. 3.
  4. 4.
    Darema-Rogers, F., George, D.A., Norton, V.A., Pfister, G.F.: Single-program-multiple-data computational model for EPEX/ FORTRAN. Technical Report RC 11552, IBM T.J. Watson Research Center, November 1986 Google Scholar
  5. 5.
    Edjlali, G., Sussman, A., Saltz, J.: Interoperability of data parallel runtime libraries. In: Proceedings of the 11th International Parallel Processing Symposium, April 1997 Google Scholar
  6. 6.
    Fide, S., Jenks, S.: Cluster-to-cluster communications: a consistent communication paradigm for distributed clusters. In: Proceedings of the 16th IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 2004), Cambridge, MA, November 2004 Google Scholar
  7. 7.
    National Partnership for Advanced Computational Infrastructure (NPACI): NPACI ROCKS Cluster Distribution: User’s Guide, 2002, http://rocks.npaci.edu
  8. 8.
    MPI Forum: Message Passing Interface (MPI)-2.0 Standard Document, http://www.mpi-forum.org
  9. 9.
    Foster, I.: What is the grid? A Three Point Checklist. GRIDToday, July 2002 Google Scholar
  10. 10.
    Foster, I., Karonis, N.: A grid-enabled MPI: message passing in heterogeneous distributed computing systems. In: Proceedings 1998 Supercomputing Conference, 1998 Google Scholar
  11. 11.
    Foster, I., Karonis, N., Kesselman, C., Koenig, G., Tuecke, S.: A secure communications infrastructure for high performance distributed computing. In: Proceedings 6th IEEE Symposium on High Performance Distributed Computing, 1997 Google Scholar
  12. 12.
    Foster, I., Kesselman, C.: Globus: a metacomputing infrastructure toolkit. Int. J. Supercomp. Appl. 11(2), 115–128 (1997) CrossRefGoogle Scholar
  13. 13.
    Foster, I., Kesselman, C.: The Grid: Blueprint for a New Computing Infrastructure. Kaufmann, San Francisco (1998) Google Scholar
  14. 14.
    Geist, G.A., Kohl, J.A., Papadopoulos, P.M.: CUMULVS: providing fault-tolerance, visualization and steering of parallel applications. In: Proceedings of the Environment and Tools for Parallel Scientific Computing Workshop, August 1996 Google Scholar
  15. 15.
    Gropp, W., Huss-Lederman, S., Lumsdaine, A., Lusk, E., Nitzberg, B., Saphir, W., Snir, M.: The MPI Extensions. MPI—The Complete Reference, vol. 2. MIT Press, Cambridge (1998) Google Scholar
  16. 16.
    Gropp, W., Lusk, E., Skjellum, A.: Using MPI: Portable Parallel Programming with the Message Passing Interface, 2nd edn. MIT Press, Cambridge (1999) CrossRefGoogle Scholar
  17. 17.
    Gropp, W., Lusk, E., Thakur, R.: Using MPI-2: Advanced Features of the Message Passing Interface. MIT Press, Cambridge (1999) CrossRefGoogle Scholar
  18. 18.
    Myricom Inc.: Myricom Home Page, http://www.myri.com
  19. 19.
    Karonis, N., Toonen, B., Foster, I.: MPICH-G2: a grid-enabled implementation of the message passing interface. J. Parallel Distributed Comput. 63(5), 551–563 (2003) MATHCrossRefGoogle Scholar
  20. 20.
    Keahey, K., Fasel, P.K., Mniszewski, S.M.: PAWS: collective interactions and data transfers. In: The 10th IEEE International Symposium on High Performance Distributed Computing, August 2001 Google Scholar
  21. 21.
    Klasky, S., Ethier, S., Lin, Z., Martins, K., McCune, D., Samtaney, R.: Grid-based parallel data streaming implemented for the gyrokinetic toroidal code. In: Proceedings SC2003: High Performance Networking and Computing. IEEE Computer Society Press, Washington (2003) Google Scholar
  22. 22.
    Argonne National Laboratory: MPICH: A Portable MPI Implementation, http://www-unix.mcs.anl.gov/mpi/mpich/
  23. 23.
    LAM/MPI Parallel Computing, http://www.lam-mpi.org
  24. 24.
    Muller, M., Hess, M., Gabriel, E.: Grid enabled MPI solutions for clusters. In: Proceedings 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID’03), Tokyo, Japan, May 2003. IEEE Computer Society Press, Washington (2003) Google Scholar
  25. 25.
    Papadopoulos, P.M., Kohl, J.A., Semeraro, B.D.: CUMULVS: extending a generic steering and visualization middleware for application fault-tolerance. In: Proceedings of the 31st Annual Hawaii International Conference on System Sciences, January 1998 Google Scholar
  26. 26.
  27. 27.
    Petrini, F., Feng, W.C., Hoisie, A., Coll, S., Frachtenberg, E.: The quadrics network: high-performance clustering technology. IEEE Micro, 46–57 (2002) Google Scholar
  28. 28.
    Plesums, C.: Introduction to Workflow, 2003, http://www.e-workflow.org/bookstore/introduction_to_workflow02.pdf
  29. 29.
    The Globus Project, http://www.globus.org
  30. 30.
    Ranganathan, M., Acharya, A., Edjlali, G., Sussman, A., Saltz, J.: Runtime coupling of data-parallel programs. In: Proceedings of the 1996 International Conference on Supercomputing, May 1996 Google Scholar
  31. 31.
    Scalable Coherent Interface, August 1993. IEEE Standard 1596-1992 Google Scholar
  32. 32.
    Smarr, L.L., Chien, A.A., DeFanti, T., Leigh, J., Papadopoulos, P.M.: The OptIPuter. Commun. ACM 46(11), 58–67 (2003) CrossRefGoogle Scholar
  33. 33.
    Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: The MPI Core, 2nd edn. MPI—The Complete Reference, vol. 1. MIT Press, Cambridge (1998) Google Scholar
  34. 34.
    Taflove, A.: Computational Electrodynamics: The Finite-Difference Time-Domain Method. Artech House, Norwood (1995) MATHGoogle Scholar
  35. 35.
    Finite-Difference Time-Domain Code, http://www.borg.umn.edu/toyfdtd/

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of California, IrvineIrvineUSA

Personalised recommendations