Universal Execution of Parallel Processes: Penetrating NATs over the Grid
Today, clusters are very important computing resources and many computing centers manage their clusters in private networks. But parallel programs may not work in private clusters. Because hosts in private clusters are not globally reachable, hosts behind different private clusters cannot be reached directly in order to communicate. It will certainly be a huge loss of resources if private clusters are excluded from the computing due to this reason. There has been much research on this issue, but most of them concentrate on user-level relaying because it is a general and easily-implementable solution. However, even well-implemented, user-level solutions result in much longer communication latency than kernel-level solutions. This paper adopted a novel kernel-level solution and applied it to MPICH-G2. Our scheme is generally applicable, simple and efficient. The experimental results show that our scheme incurs very little overhead except when small messages are transmitted. That is, it supports a more universal computing environment by including private clusters with remarkably little overhead.
Unable to display preview. Download preview PDF.
- 1.Park, K., Park, S.-Y., Kwon, O.-Y., Park, H.-W.: MPICH-GP: A private-IP-enabled MPI over grid environments. In: Cao, J., Yang, L.T., Guo, M., Lau, F. (eds.) ISPA 2004. LNCS, vol. 3358, pp. 469–473. Springer, Heidelberg (2004)Google Scholar
- 2.Gabriel, E., Resch, M., Beisel, T., Keller, R.: Distributed computing in a heterogenous computing environment. In: EuroPVMMPI (1998)Google Scholar
- 3.Dongarra, J., Fagg, G.E., Geist, A., Kohl, J.A., Papadopoulos, P.M., Scott, S.L., Sunderamand, V.S., Magliardi, M.: HARNESS: Heterogeneous Adaptable Reconfigurable NEtworked SystemS. In: HPDC (1998)Google Scholar
- 4.Frumkin, M., Jin, H., Yan, J.: Implementation of NAS Parallel Benchmarks in High Performance Fortran. NAS Techinical Report NAS-98-009 (1998)Google Scholar
- 5.Srisuresh, P., Holdrege, M.: IP network address translator (NAT) terminology and considerations. RFC 2663 (1999)Google Scholar
- 6.Ford, B., Srisuresh, P., Kegel, D.: Peer-to-peer communication across middleboxes. Internet Draft draftford- midcom-p2p-01, Internet Engineering Task Force, Work in progress (2003)Google Scholar
- 7.Rosenberg, J.: Traversal Using Relay NAT (TURN). draftrosenberg- midcom-turn-04 (2004)Google Scholar
- 8.Ford, B., Srisuresh, P., Kegel, D.: Peer-to-Peer Communication Across Network Address Translators. In: USENIX 2005, pp. 179–192 (2005)Google Scholar
- 9.Muller, M., Hess, M., Gabriel, E.: Grid enabled MPI solutions for Clusters. In: CCGRID’03, pp. 18–24 (2003)Google Scholar
- 10.Guha, S., Francis, P.: Characterization and Measurement of TCP Traversal through NATs and Firewalls. In: IMC (2005)Google Scholar
- 11.OpenMP Architecture Review Board. OpenMP, http://www.openmp.org
- 12.Carpenter, B., Getov, V., Judd, G., Skjellum, T., Fox, G.: MPJ: MPI-like Message Passing for Java. Concurrency: Practice and Experience 12(11) (2000)Google Scholar