# Graph Partitioning Algorithms with Applications to Scientific Computing

• Alex Pothen
Chapter
Part of the ICASE/LaRC Interdisciplinary Series in Science and Engineering book series (ICAS, volume 4)

## Abstract

Identifying the parallelism in a problem by partitioning its data and tasks among the processors of a parallel computer is a fundamental issue in parallel computing. This problem can be modeled as a graph partitioning problem in which the vertices of a graph are divided into a specified number of subsets such that few edges join two vertices in different subsets. Several new graph partitioning algorithms have been developed in the past few years, and we survey some of this activity.

We describe the terminology associated with graph partitioning, the complexity of computing good separators, and graphs that have good separators. We then discuss early algorithms for graph partitioning, followed by three new algorithms based on geometric, algebraic, and multilevel ideas. The algebraic algorithm relies on an eigenvector of a Laplacian matrix associated with the graph to compute the partition. The algebraic algorithm is justified by formulating graph partitioning as a quadratic assignment problem. We list several papers that describe applications of graph partitioning to parallel scientific computing and other applications. Finally we discuss the application of graph partitioning to obtain nested dissection orderings for solving sparse linear systems of equations in parallel.

## Preview

Unable to display preview. Download preview PDF.

### References

1. Agrawal, A., Klein, P.N., and Ravi, R., 1993. “Cutting down on fill using nested dissection: Provably good elimination orderings,” in Graph Theory and Sparse Matrix Computation, A. George, J.R. Gilbert, and J.W.H. Liu, eds., IMA Volumes in Mathematics and its Applications 56, Springer Verlag, Berlin, pp. 31–55.Google Scholar
2. Alon, N. and Milman, V., 1985. “λ1, isoperimetric inequalities for graphs, and superconcentrators,” J. of Combin. Theory, Series B 38, pp. 73–88.Google Scholar
3. Alpert, C.J. and Kahng, A., 1995. “Recent directions in netlist partitioning: A survey,” Integration: The VLSI Journal 19, pp. 1–81.
4. Arun, K.S. and Rao, V.B., 1993. “Constructive heuristics and lower bounds for graph partitioning based on a principal components approximation,” SIAM J. Matrix Anal. Appl. 14, pp. 991–1015.
5. Ashcraft, C. and Liu, J.W.H., 1995. “Using domain decomposition to find graph bisectors,” Tech. Report CS-95-08, Department of Computer Science, York University, North York, Ontario, Canada, M3J 1P3.Google Scholar
6. Ashcraft, C. and Liu, J.W.H., 1996. “Robust ordering of sparse matrices using multisection”, Tech. Report CS-96-01, Department of Computer Science, York University, North York, Ontario, Canada, M3J 1P3.Google Scholar
7. Aspvall, B. and Gilbert, J.R., 1984. “Graph coloring using eigenvalue decomposition,” SIAM J. Alg. Disc. Meth. 5, pp. 526–538.
8. Barnard, S.T., Pothen, A., and Simon, H.D., 1995. “A spectral algorithm for envelope reduction of sparse matrices,” Numer. Lin. Alg. Applics. 2, pp. 317–334.
9. Barnard, S.T. and Simon, H.D., 1994. “A fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems,” Concurrency: Practice and Experience 6, pp. 101–117.Google Scholar
10. Barnes, E.R., 1982. “An algorithm for partitioning the nodes of a graph,” SIAM J. Alg. Disc. Meth. 3, pp. 541–550.
11. Berger, M. J., and Bokhari, S., 1987. “A partitioning strategy for nonuniform problems on multiprocessors,” IEEE Trans. Computers C-36, pp. 570–580.Google Scholar
12. Biggs, N., 1993. Algebraic Graph Theory, second edition, Cambridge University Press, Cambridge.Google Scholar
13. Bollobás, B., 1979. Graph Theory: An introductory course, Springer Verlag, Berlin.
14. Boppana, R.B., 1987. “Eigenvalues and graph bisection: An average case analysis,” Proc. 28th Ann. Symp. Foundations of Computer Science, pp. 280–285.Google Scholar
15. Bui, T.N., Fukuyama, F., and Jones, C., 1994. “The planar vertex separator problem: Complexity and algorithms,” Manuscript, Department of Computer Science, Pennsylvania State University, Harrisburg, PA.Google Scholar
16. Bui, T.N. and Jones, C., 1992. “Finding good approximate vertex and edge partitions is NP-hard,” Inf. Process. Letters 42, pp. 153–159.
17. Bui, T.N. and Jones, C., 1993. “A heuristic for reducing fill-in in sparse matrix factorization,” Proc. Sixth SIAM Conf. on Parallel Processing for Scientific Computing, SIAM, Philadelphia, pp. 445–452.Google Scholar
18. Bui, T.N. and Moon, B.R., 1993. “A genetic algorithm for a special class of the quadratic assignment problem,” in The Quadratic Assignment and Related Problems, P. Pardalos and H. Wolkow-icz, eds., DIMACS Series on Discrete Mathematics and Theoretical Computer Science, AMS, Providence.Google Scholar
19. Chan, T.F., Ciarlet, P., Jr., and Szeto, W.K., 1993. “On the near optimality of the recursive spectral bisection method for graph partitioning,” Manuscript, Department of Computer Science, University of California at Los Angeles.Google Scholar
20. Chan, T.F., Gilbert, J.R., and Teng, S.-H., 1994. “Geometric spectral bisection,” Preprint, Xerox Palo Alto Research Center.Google Scholar
21. Chan, T.F. and Mathew, T.P., 1994. “Domain decomposition algorithms,” Acta Numerica 3, pp. 61–143.
22. Chrisochoides, N., Mansour, N., and Fox, G., 1994. “Mapping algorithms and software environment for data parallel PDE iterative solvers,” J. Par. Dist. Comp. 21, pp. 75–95.Google Scholar
23. Chrisochoides, N., Mansour, N., and Fox, G., 1993. “Performance evaluation of data mapping algorithms for parallel single-phase iterative PDE solvers,” Preprint, Northeast Parallel Architectures Center, Syracuse University.Google Scholar
24. Chu, E.G., George, J.A., Liu, J.W., and Ng, E.G., 1984. “User’s guide for SPARSPAK-A: Waterloo sparse linear equations package,” Tech. Report CS-84-36, Computer Science, University of Waterloo, Ontario, Canada.Google Scholar
25. Ciarlet, P., Jr. and Lamour, F., April 1994. “Spectral partitioning methods and greedy partitioning methods: A comparison on finite element graphs,” Tech. Report 94-9, Department of Mathematics, University of California at Los Angeles.Google Scholar
26. Ciarlet, P., Jr., Lamour, F., and Smith, B.F., 1995. “On the influence of the partitioning scheme on the efficiency of overlapping domain decomposition methods,” Fifth Symp. Frontiers of Massively Parallel Computation, IEEE Press, Los Alamitos, pp. 375–383.Google Scholar
27. Cvetkovic, D., Doob, M., and Sachs, H., 1980. Spectra of Graphs, Academic Press, New York.Google Scholar
28. Cvetkovic, D.M., Doob, M., Gutman, I., and Torgasev, A., 1988. “Recent Results in the Theory of Graph Spectra”, Annals of Discrete Mathematics 36, North Holland.Google Scholar
29. Dagum, L., 1995. “Automatic partitioning of unstructured grids into connected components,” Preprint, Computer Science Corporation, NASA Ames Research Center, Moffett Field, CA.Google Scholar
30. Das, R., Mavriplis, D.J., Saltz, J., Gupta, S., and Ponnusamy, R., 1992. “The design and implementation of a parallel unstructured Euler solver using software primitives,” Proc. AIAA 30th Aerospace Sciences Meeting, Paper AIAA-92-0562.Google Scholar
31. Donath, W.E. and Hoffman, A.J., 1973. “Lower bounds for the partitioning of graphs,” IBM Journal of Research and Development 17, pp. 420–425.
32. Duff, I.S., Erisman, A.M., and Reid, J.K., 1986. Direct Methods for Sparse Matrices, Clarendon Press, Oxford.
33. Duff, I.S., Grimes, R.G., and Lewis, J.G., 1989. “Sparse matrix test problems,” ACM Trans. on Math. Soft. 15, pp. 1–14.
34. Falkner, J., Rendl, F., and Wolkowicz, H., 1994. “A computational study of graph partitioning,” Math. Programming 66, no.2 Ser. A, pp. 211–239.
35. Farhat, C., Lanteri, S., and Simon, H.D., 1995. “TOP/DOMDEC: A software tool for mesh partitioning and parallel processing,” Computing Systems in Engineering 6, pp. 13–26.Google Scholar
36. Farhat, C. and Roux, F.X., 1994. “Implicit parallel processing in structural mechanics,” Computational Mechanics Advances 2, pp. 1–124.
37. Fiduccia, C. and Mattheyses, R., 1982. “A linear time heuristic for improving network partitions,” ACM — IEEE 19th Design Automation Conf., IEEE Press, Los Alamitos, pp. 175–181.Google Scholar
38. Fiedler, M., 1973. “Algebraic connectivity of graphs,” Czech. Math. J. 23, pp. 298–305.
39. Fiedler, M., 1975. “A property of eigenvectors of non-negative symmetric matrices and its applications to graph theory,” Czech. Math. J. 25, pp. 619–633.
40. Garey, M.R. and Johnson, D.S., 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness, W.H. Freeman.Google Scholar
41. George, A. and Liu, J.W.H., 1981. Computer Solution of Large Sparse Positive Definite Systems, Prentice Hall, Englewood Cliffs.
42. George, A., 1973. “Nested dissection of a regular finite element mesh,” SIAM J. Numer. Anal 10, pp. 345–363.
43. George, A. and Liu, J.W.H., 1989. “The evolution of the minimum degree algorithm,” SIAM Review 31, pp. 1–19.
44. George, A. and Pothen, A., 1995. “Analysis of the spectral approach to envelope reduction via a quadratic assignment formulation,” SIAM J. Matrix Anal. Appl. (to appear).Google Scholar
45. Gilbert, J.R., 1980. “Graph Separator Theorems and Sparse Gaussian Elimination,” PhD thesis, Stanford University.Google Scholar
46. Gilbert, J.R., Miller, G.L., and Teng, S.-H., 1994. “Geometric mesh partitioning: Implementation and experiments,” Tech. Report CSL-94-13, Xerox Palo Alto Research Center.Google Scholar
47. Gilbert, J.R. and Zmijewski, E., 1987. “A parallel graph partitioning algorithm for a message-passing multiprocessor,” International Journal of Parallel Programming 16, pp. 427–449.
48. Goehring, T. and Saad, Y., 1994. “Heuristic algorithms for automatic graph partitioning,” Tech. Report, Department of Computer Science, University of Minnesota, Minneapolis.Google Scholar
49. Golub, G.H. and Van Loan, C.F., 1989. Matrix Computations, second edition, Johns Hopkins University Press, Baltimore.
50. Guattery, S. and Miller, G., 1995. “On the performance of spectral graph partitioning methods,” 6th ACM — SIAM Symp. on Discrete Algorithms, San Francisco, CA, ACM-SIAM, Philadelphia, pp. 233–242.Google Scholar
51. Hagen, L. and Kahng, A., 1992. “New spectral methods for ratio cut partitioning and clustering,” IEEE Transactions on Computer-Aided Design 11, pp.1074–1085.Google Scholar
52. Hall, K.M., 1970. “An r-dimensional quadratic placement algorithm,” Management Science 17, pp. 219–229.
53. Hammond, S., 1992. “Mapping unstructured grid computations to massively parallel computers,” PhD thesis, Rensselaer Polytechnic Institute, Troy, NY, RIACS Report 92-14.Google Scholar
54. Heath, M.T., Ng, E.G.Y., and Peyton, B.W., 1991. “Parallel algorithms for sparse linear systems,” SIAM Review 33, pp. 420–460.
55. Heath, M.T. and Raghavan, P., 1995. “A Cartesian parallel nested dissection algorithm,” Tech. Report 92-1772, Department of Computer Science, University of Illinois, Urbana.Google Scholar
56. Hendrickson, B. and Leland, R., 1993. “The CHACO user’s guide,” Tech. Report 93-2339, Sandia National Labs, Albuquerque, NM.Google Scholar
57. Hendrickson, B. and Leland, R., 1993. “Multidimensional load balancing,” Tech. Report 93-0074, Sandia National Labs, Albuquerque, NM.Google Scholar
58. Hendrickson, B. and Leland, R., 1993. “A multilevel algorithm for partitioning graphs,” Tech. Report 93-1301, Sandia National Labs, Albuquerque, NM.Google Scholar
59. Hendrickson, B. and Leland, R., 1995. “An improved spectral graph partitioning algorithm for mapping parallel computations,” SIAM J. Stat Comput. 16, pp. 452–469.
60. Hendrickson, B. and Rothberg, E., 1996. “Improving the runtime and quality of nested dissection ordering,” Preprint. Silicon Graphics, Inc., Mountain View, CA94043.Google Scholar
61. Hodgson, D.C. and Jimack, P.K., 1993. “Efficient mesh partitioning for parallel PDE solvers on distributed memory machines,” Tech. Report 93-1, School of Computer Studies, University of Leeds.Google Scholar
62. Hsieh, S.H., Paulino, G.H., and Abel, J.F., 1993. “Recursive spectral algorithms for automatic domain partitioning in parallel finite element analysis,” Preprint, Civil Engineering Department, Cornell University.Google Scholar
63. Johan, Z., 1992. “Data parallel finite element techniques for large-scale Computational Fluid Dynamics,” PhD thesis, Stanford University.Google Scholar
64. Johan, Z., Mathur, K.K., Johnsson, S.L., and Hughes, T.J.R., 1994. “Parallel implementation of recursive spectral bisection on the Connection Machine CM-5 system,” Tech. Report 07-94, Center for Research in Computing Technology, Harvard University.Google Scholar
65. Johnson, D., Aragon, C., McGeoch, L., and Schevon, C., 1989. “Optimization by simulated annealing: An experimental evaluation, Part I, Graph Partitioning,” Operations Research 37, pp. 865–892.
66. Jones, C., 1992. “Vertex and edge partitions of graphs,” PhD thesis, The Pennsylvania State University.Google Scholar
67. Karypis, G. and Kumar, V., 1995. “A fast and high quality multilevel scheme for partitioning irregular graphs,” Tech. Report 95-035, Computer Science Department, University of Minnesota.Google Scholar
68. Karypis, G. and Kumar, V., 1995. “Parallel multilevel graph partitioning,” Tech. Report 95-036, Computer Science Department, University of Minnesota.Google Scholar
69. Karypis, G. and Kumar, V., 1995. “Analysis of multilevel graph partitioning,” Tech. Report 95-037, Gomputer Science Department, University of Minnesota.Google Scholar
70. Kernighan, B.W. and Lin, S., 1970. “An efficient heuristic procedure for partitioning graphs,” The Bell System Technical J. 49, pp. 291–307.
71. Keyes, D.E., 1992. “Domain decomposition: A bridge between nature and parallel computers,” in Adaptive Multilevel and Hierarchical Computational Strategies, A.K. Noor, ed., ASME, New York, pp. 293–334.Google Scholar
72. Keyes, D.E., Saad, Y., and Truhlar, D.G., eds., 1995. Domain-based Parallelism and Problem Decomposition Methods in Science and Engineering, SIAM, Philadelphia.
73. Kumfert, G. and Pothen, A., 1995. “A multilevel nested dissection algorithm,” unpublished work.Google Scholar
74. Leete, C.A., Peyton, B.W., and Sincovec, R.F., 1993. “Toward a parallel recursive spectral bisection mapping tool,” in Proc. Sixth SIAM Conf. on Parallel Processing, SIAM, Philadelphia, pp. 923–928.Google Scholar
75. Leighton, F.T. and Rao, S., 1988. “An approximate max-flow min-cut theorem for uniform multicommodity flow problems with applications to approximation algorithms,” in Proc. 29th Ann. Symp. on Foundations of Computer Science, IEEE, pp. 422–431.Google Scholar
76. LeTallec, P.J., 1994. “Domain decomposition methods in computational mechanics,” Computational Mechanics Advances 2, pp. 121–220.
77. Lewis, J.G., Peyton, B.W., and Pothen, A., 1989. “A fast algorithm for reordering sparse matrices for parallel factorization,” SIAM J. Sci. Stat. Comput. 10, pp. 1146–1173.
78. Lipton, R.J., Rose, D.J., and Tarjan, R.E., 1979. “Generalized nested dissection,” SIAM J. Numer. Anal. 16, pp. 346–358.
79. Lipton, R.J. and Tarjan, R.E., 1979. “A separator theorem for planar graphs,” SIAM J. Appl. Math. 36, pp. 177–189.
80. Liu, J.W.H., 1985. “Modification of the minimum degree algorithm by multiple elimination,” ACM Trans. on Math. Soft. 11, pp. 141–153.
81. Liu, J.W.H., 1989. “The minimum degree ordering with constraints,” SIAM J. Sci. Stat. Comp. 10, pp. 198–219.Google Scholar
82. Liu, J.W.H., 1990. “The role of elimination trees in sparse factorization,” SIAM J. Matrix Anal. Appl. 11, pp. 134–172.
83. Miller, G.L., Teng, S.-H., Thurston, W., and Vavasis, S.A., 1993. “Automatic mesh partitioning,” in Graph Theory and Sparse Matrix Computation, A. George, J.R. Gilbert, and J.W.H. Liu, eds., IMA Volumes in Mathematics and its Applications 56, Springer Verlag, Berlin, pp. 57–84.Google Scholar
84. Mohar, B., 1991. “The Laplacian spectrum of graphs,” Graph Theory Combinatorics, and Applications, John Wiley, New York, pp. 871–898.Google Scholar
85. Mohar, B. and Poljak, S., 1992. “Eigenvalues in combinatorial optimization,” Preprint, Department of Mathematics, University of Ljubljana, Jadranska 19, 61 111, Ljubljana, Slovenia.Google Scholar
86. Nour-Omid, B., Raefsky, A., and Lyzenga, G., 1986. “Solving finite element equations on concurrent computers,” in Parallel Computations and their Impact on Mechanics, A.K. Noor, ed., ASME, New York, pp. 209–227.Google Scholar
87. Oden, J.T., Patra, A., and Feng, Y., 1994. “Domain decomposition for adaptive hp finite element methods,” in Domain Decomposition Methods in Science and Engineering, D.E. Keyes and J. Xu, eds., Contemporary Mathematics 180, AMS, Providence pp. 295–301.Google Scholar
88. Ozturan, C., 1995. “Distributed environment and load balancing for adaptive unstructured meshes,” PhD thesis, Rensselaer Polytechnic Institute, Troy, NY.Google Scholar
89. Parlett, B.N., 1980. The Symmetric Eigenvalue Problem, Prentice Hall, Englewood Cliffs.
90. Pothen, A., 1994. “An analysis of spectral graph partitioning via quadratic assignment problems,” in Domain Decomposition Methods in Science and Engineering, D.E. Keyes and J. Xu, eds., Contemporary Mathematics 180, AMS, Providence, pp. 105–110.Google Scholar
91. Pothen, A. and Fan, C.-J., 1990. “Computing the block triangular form of a sparse matrix,” ACM Trans. on Math. Soft. 16, pp. 303–324.
92. Pothen, A., Rothberg, E., Simon, H.D., and Wang, L., 1994. “Parallel sparse Cholesky factorization with spectral nested dissection ordering,” in Proc. of the Fifth SIAM Conf on Applied Linear Algebra, J.G.L. et al., eds., SIAM, Philadelphia, pp. 418–422.Google Scholar
93. Pothen, A., Simon, H.D., and Liou, K.P., 1990. “Partitioning sparse matrices with eigenvectors of graphs,” SIAM J. Matrix Anal. Appl. 11, pp. 430–452.
94. Pothen, A., Simon, H.D., and Wang, L., 1992. “Spectral nested dissection,” Tech. Report CS-92-01, Computer Science, Pennsylvania State University, University Park, PA.Google Scholar
95. Pothen, A., Wang, L., Simon, H., and Barnard, S., 1992. “Towards a fast implementation of spectral nested dissection,” in Proc. Supercomputing ′92, IEEE Computer Society Press, Los Alamitos, pp. 42–51.Google Scholar
96. Rendl, F. and Wolkowicz, H., 1995. “A projection technique for partitioning the nodes of a graph,” Annals of Operations Research 58, pp. 155–179. (This paper was written in 1990, and appeared in in the special issue devoted to the Symp. on Applied Mathematical Programming and Modeling, Budapest, January1993.)
97. Roose, D. and VanDriessche, R., 1993. “Distributed memory parallel computers and Computational Fluid Dynamics,” Tech. Report TW 186, Department of Computer Science, Katholike Universiteit Leuven, Belgium.Google Scholar
98. Schramm, H. and Zowe, J., 1992. “A version of the bundle idea for minimizing a nonsmooth function: Conceptual idea, convergence analysis, numerical results,” SIAM J. Opt. 2, pp. 121–152.
99. Shephard, M.S., Flaherty, J.E., DeCougny, H.L., Özturan, C., Bot-tasso, C.L., and Beall, M., 1995. “Parallel automated adaptive procedures for unstructured meshes,” Parallel Computing in CFD, Advisory Group for Aerospace Research and Development (AGARD), Neuilly-Sur-Seine, France, Report R-807.Google Scholar
100. Simon, H.D., 1991. “Partitioning of unstructured problems for parallel processing,” Computing Systems in Engineering 2, pp. 135–148.Google Scholar
101. Smith, B.F., Bjørstad, P.E., and Gropp, W.D., 1996. “Domain Decomposition: Parallel Multilevel Algorithms for Elliptic Partial Differential Equations,” Cambridge University Press, Cambridge.
102. Suaris, P. and Kedem, G., 1988. “An algorithm for quadrisection and its application to standard cell placement,” IEEE Transactions on Circuits and Systems 35, pp. 294–303.Google Scholar
103. Spielman, D.A. and Teng, S-H., 1996, “Spectral partitioning works: Planar graphs and finite element meshes”, Draft manuscript, http://cs.berkeley.edu/~spielman/spect.html.Google Scholar
104. Vanderstraeten, D., Farhat, C., Chen, P.S., Keunings, R., and Zone, O., 1994. “A retrofit based methodology for the fast generation and optimization of large-scale mesh partitions,” Tech. Report CAS-94-18, Center for Aerospace Structures, University of Colorado, Boulder.Google Scholar
105. VanDriessche, R. and Roose, D., 1995. “A graph contraction algorithm for the fast calculation of the Fiedler vector of a graph,” Proc. Seventh SIAM Conf. on Parallel Computing for Scientific Computing, SIAM, Philadelphia, pp. 621–626.Google Scholar
106. VanDriessche, R. and Roose, D., 1994. “An efficient spectral bisection algorithm for dynamic load balancing,” Tech. Report TW 215, Department of Computer Science, Katholike Universiteit Leuven, Belgium.Google Scholar
107. Venkatakrishnan, V., Simon, H.D., and Barth, T., 1992. “A MIMD implementation of a parallel Euler solver for unstructured grids,” J. Supercomputing 6, pp. 117–127.
108. Walshaw, C. and Berzins, M., 1992. “Dynamic load balancing for PDE solvers on adaptive unstructured meshes,” Tech. Report 92-32, School of Computer Studies, University of Leeds.Google Scholar
109. Wang, L., 1994. “Spectral Nested Dissection Orderings for Parallel Sparse Matrix Factorization,” PhD thesis,, Computer Science Department, The Pennsylvania State University.Google Scholar
110. Williams, R.D., 1991. “Performance of dynamic load balancing algorithms for unstructured mesh calculations,” Concurrency: Practice and Experience 3, pp. 457–481.Google Scholar