Can MPI Be Used for Persistent Parallel Services?
MPI is routinely used for writing parallel applications, but it is not commonly used for writing long-running parallel services, such as parallel file systems or job schedulers. Nonetheless, MPI does have many features that are potentially useful for writing such software. Using the PVFS2 parallel file system as a motivating example, we studied the needs of software that provide persistent parallel services and evaluated whether MPI is a good match for those needs. We also ran experiments to determine the gaps between what the MPI Standard enables and what MPI implementations currently support. The results of our study indicate that MPI can enable persistent parallel systems to be developed with less effort and can provide high performance, but MPI implementations will need to provide better support for certain features. We also describe an area where additions to the MPI Standard would be useful.
KeywordsCollective Operation Parallel Service Parallel File System Data Distribution Scheme Aggregate Operation
Unable to display preview. Download preview PDF.
- 1.Bosilca, G., Bouteiller, A., Cappello, F., Djilali, S., Fedak, G., Germain, C., Herault, T., Lemarinier, P., Lodygensky, O., Magniette, F., Neri, V., Selikhov, A.: MPICH-V: Toward a scalable fault tolerant MPI for volatile nodes. In: Supercomputing 2002: Proceedings of the 2002 ACM/IEEE Conference on Supercomputing, pp. 1–18. IEEE Computer Society Press, Los Alamitos (2002)Google Scholar
- 2.Carns, P.H., Ligon, III.W.B., Ross, R.B., Thakur, R.: PVFS: A parallel file system for Linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, Atlanta, GA, October 2000, pp. 317–327 (2000)Google Scholar
- 3.Carns, P.: Design and analysis of a network transfer layer for parallel file systems. Master’s thesis, Clemson University, Clemson, S.C. (July 2001)Google Scholar
- 4.Carns, P.H.: Achieving Scalability in Parallel File Systems. PhD thesis, Dept. of Electrical and Computer Engineering, Clemson University, Clemson, SC (May 2004)Google Scholar
- 7.Gabriel, E., Fagg, G.E., Bosilca, G., Angskun, T., Dongarra, J. J., Squyres, J.M., Sahay, V., Kambadur, P., Barrett, B., Lumsdaine, A., Castain, R.H., Daniel, D.J., Graham, R.L., Woodall, T.S.: Open MPI: Goals, concept, and design of a next generation MPI implementation. In: Kranzlmüller, D., Kacsuk, P., Dongarra, J. (eds.) EuroPVM/MPI 2004. LNCS, vol. 3241, pp. 97–104. Springer, Heidelberg (2004)CrossRefGoogle Scholar
- 9.International Business Machines Corporation. IBM Parallel Environment for AIX 5L: MPI Subroutine Reference, 3rd edn. (April 2005)Google Scholar
- 10.Message Passing Interface Forum. MPI-2: Extensions to the message-passing interface (July 1997), http://www.mpi-forum.org/docs/docs.html
- 12.The PVFS2 parallel file system, http://www.pvfs.org/pvfs2