What MPI Could (and Cannot) Do for Mesh-Partitioning on Non-homogeneous Networks

  • Guntram Berti
  • Jesper Larsson Träff
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4192)


We discuss the mesh-partitioning load-balancing problem for non-homogeneous communication systems, and investigate whether the MPI process topology functionality can aid in solving the problem. An example kernel shows that specific communication patterns can benefit substantially from a non-trivial MPI topology implementation, achieving improvements beyond a factor of five for certain system configurations. Still, the topology functionality lacks expressivity to deal effectively with the mesh-partitioning problem. A mild extension to MPI is suggested, which, however, still cannot exclude possibly sub-optimal partitioning results. Solving instead the mesh-partitioning problem outside of MPI requires knowledge of the communication system. We discuss ways in which such could be provided by MPI in a portable way. Finally, we formulate and discuss a more general affinity scheduling problem.


Communication Graph Host Graph Processor Mapping Topology Functionality Processor Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Devine, K.D., Boman, E.G., Heaphy, R.T., Hendrickson, B.A., Teresco, J.D., Faik, J., Flaherty, J.E., Gervasio, L.G.: New challenges in dynamic load balancing. Appl. Numer. Math. 52(2–3), 133–152 (2005)zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Fiduccia, C.M., Mattheyses, R.M.: A linear-time heuristic for improving network partitions. In: 19th ACM/IEEE Design Automation Conference (DAC), pp. 175–181 (1982)Google Scholar
  3. 3.
    Garey, M.R., Johnson, D.S., Stockmeyer, L.J.: Some simplified NP-complete graph problems. Theoretical Computer Science 1(3), 237–267 (1976)zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Hendrickson, B., Kolda, T.G.: Graph partitioning models for parallel computing. Parallel Computing 26, 1519–1534 (2000)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Hendrickson, B., Leland, R.: A multi-level algorithm for partitioning graphs. In: Proceedings of the 1995 ACM/IEEE Supercomputing Conference (1995)Google Scholar
  6. 6.
    Hendrickson, B., Leland, R., Driessche, R.V.: Skewed graph partitioning. In: Proc. 8th SIAM Conference on Parallel Processing for Scientific Computing (1997)Google Scholar
  7. 7.
  8. 8.
    Kernighan, B.W., Lin, S.: An efficient heuristic procedure for partitioning graphs. Bell System Technical Journal 49, 291–307 (1970)Google Scholar
  9. 9.
    Moulitsas, I., Karypis, G.: Architecture aware partitioning algorithms. Technical Report DCT Research Report 2006/02, Digital Technology Center, University of Minnesota (January 2006)Google Scholar
  10. 10.
    Pellegrini, F., Roman, J.: SCOTCH: A software package for static mapping by dual recursive bipartitioning of process and architecture graphs. In: Liddell, H., Colbrook, A., Hertzberger, B., Sloot, P.M.A. (eds.) HPCN-Europe 1996. LNCS, vol. 1067, pp. 493–498. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  11. 11.
    Savage, J.E., Wloka, M.G.: Parallelism in graph-partitioning. Journal of Parallel and Distributed Computing 13, 257–272 (1991)CrossRefMathSciNetGoogle Scholar
  12. 12.
    Snir, M., Otto, S., Huss-Lederman, S., Walker, D., Dongarra, J.: MPI – The Complete Reference. In: The MPI Core, 2nd edn., vol. 1. MIT Press, Cambridge (1998)Google Scholar
  13. 13.
    Träff, J.L.: Implementing the MPI process topology mechanism. In: Supercomputing (2002),
  14. 14.
    Träff, J.L.: SMP-aware message passing programming. In: Eigth International Workshop on High-level Parallel Programming Models and Supportive Environments (HIPS 2003), pp. 56–65 (2003)Google Scholar
  15. 15.
    Valcke, S., Declat, D., Redler, R., Ritzdorf, H., Vogelsang, R., Bourcier, P.: The PRISM coupling and I/O system. In: Proceedings of VECPAR 2004 (2004)Google Scholar
  16. 16.
    Walshaw, C., Cross, M.: Multilevel mesh partitioning for heterogeneous communication networks. Future Generation Comput. Syst. 17(5), 601–623 (2001)CrossRefGoogle Scholar
  17. 17.
    Zoltan: Data-management services for parallel applications,

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Guntram Berti
    • 1
  • Jesper Larsson Träff
    • 1
  1. 1.C&C Research LaboratoriesNEC Europe Ltd.Sankt AugustinGermany

Personalised recommendations