Abstract
Structured sparse matrices can greatly benefit parallel numerical methods in terms of parallel performance and convergence. In this chapter, we present combinatorial models for obtaining several different sparse matrix forms. There are four basic forms we focus on: singly-bordered block-diagonal form, doubly-bordered block-diagonal form, nonempty off-diagonal block minimization, and block diagonal with overlap form. For each of these forms, we first present the form in detail and describe what goals are sought within the form, and then examine the combinatorial models that attain the respective form while targeting the sought goals, and finally explain in which aspects the forms benefit certain parallel numerical methods and their relationship with the models. Our work focuses especially on graph and hypergraph partitioning models in obtaining the mentioned forms. Despite their relatively high preprocessing overhead compared to other heuristics, they have proven to model the given problem more accurately and this overhead can be often amortized due the fact that matrix structure does not change much during a typical numerical simulation. This chapter presents a number of models and their relationship with parallel numerical methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Acer, S., Aykanat, C.: Reordering sparse matrices into block-diagonal column-overlapped form. Journal of Parallel and Distributed Computing 140, 99–109 (2020)
Acer, S., Kayaaslan, E., Aykanat, C.: A recursive bipartitioning algorithm for permuting sparse square matrices into block diagonal form with overlap. SIAM Journal on Scientific Computing 35(1), C99–C121 (2013)
Alpert, C.J., Kahng, A.B.: Recent directions in netlist partitioning: a survey. Integr. VLSI J. 19, 1–81 (1995)
Amestoy, P., Davis, T., Duff, I.: An approximate minimum degree ordering algorithm. SIAM Journal on Matrix Analysis and Applications 17(4), 886–905 (1996)
Aykanat, C., Pinar, A., Çatalyürek, U.V.: Permuting sparse rectangular matrices into block-diagonal form. SIAM J. Sci. Comput. 25, 1860–1879 (2004)
Bjorck, A.: Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics (1996)
Brandstädt, A., Le, V.B., Spinrad, J.P.: Graph Classes: A Survey. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1999)
Bui, T.N., Jones, C.: Finding good approximate vertex and edge partitions is np-hard. Inf. Process. Lett. 42(3), 153–159 (1992)
Bui, T.N., Jones, C.: A heuristic for reducing fill-in in sparse matrix factorization. In: Proceedings of the 6th SIAM Conference on Parallel Processing for Scientific Computing, pp. 445–452. Society for Industrial and Applied Mathematics (1993)
Catalyurek, U., Aykanat, C.: Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication. IEEE Trans. Parallel Distrib. Syst. 10, 673–693 (1999)
Catalyurek, U., Aykanat, C., Kayaaslan, E.: Hypergraph partitioning-based fill-reducing ordering for symmetric matrices. SIAM Journal on Scientific Computing 33(4), 1996–2023 (2011)
Çatalyürek, U.V., Aykanat, C.: Patoh: partitioning tool for hypergraphs. Tech. rep., Department of Computer Engineering, Bilkent University (1999)
Cong, J., Lubio, W., Shivakumur, N.: Multi-way VLSI circuit partitioning based on dual net representation. In: IEEE/ACM International Conference on Computer-Aided Design, pp. 56–62 (1994)
Dantzig, G.B., Wolfe, P.: Decomposition principle for linear programs. Oper. Res. 8(1), 101–111 (1960)
Ferris, M.C., Horn, J.D.: Partitioning mathematical programs for parallel solution. Mathematical Programming 80(1), 35–61 (1998)
Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)
George, A.: Nested dissection of a regular finite element mesh. SIAM Journal on Numerical Analysis 10(2), 345–363 (1973)
George, A., Liu, J.W.: Computer Solution of Large Sparse Positive Definite. Prentice Hall Professional Technical Reference (1981)
Golub, G., Sameh, A.H., Sarin, V.: A parallel balance scheme for banded linear systems. Numerical Linear Algebra with Applications 8(5), 285–299 (2001)
Hendrickson, B., Leland, R.: The Chaco user’s guide version 2.0 (1995)
Hendrickson, B., Leland, R.: A multilevel algorithm for partitioning graphs. In: Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM), Supercomputing ’95. ACM, New York, NY, USA (1995)
Hendrickson, B., Rothberg, E.: Improving the run time and quality of nested dissection ordering. SIAM J. Sci. Comput. 20(2), 468–489 (1998)
Kahou, G.A.A., Grigori, L., Sosonkina, M.: A partitioning algorithm for block-diagonal matrices with overlap. Parallel Comput. 34(6–8), 332–344 (2008)
Kahou, G.A.A., Kamgnia, E., Philippe, B.: An explicit formulation of the multiplicative Schwarz preconditioner. Applied Numerical Mathematics 57(11), 1197–1213 (2007). Numerical Algorithms, Parallelism and Applications (2)
Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)
Karsavuran, M.O., Akbudak, K., Aykanat, C.: Locality-aware parallel sparse matrix-vector and matrix-transpose-vector multiplication on many-core processors. IEEE Transactions on Parallel and Distributed Systems 27(6), 1713–1726 (2016)
Karypis, G., Aggarwal, R., Kumar, V., Shekhar, S.: Multilevel hypergraph partitioning: applications in VLSI domain. IEEE Trans. Very Large Scale Integr. Syst. 7, 69–79 (1999)
Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)
Kayaaslan, E., Pinar, A., Çatalyürek, U., Aykanat, C.: Partitioning hypergraphs in scientific computing applications through vertex separators on graphs. SIAM J. Sci. Comput. 34(2), 970–992 (2012)
Kou, L.T., Stockmeyer, L.J., Wong, C.K.: Covering edges by cliques with regard to keyword conflicts and intersection graphs. Commun. ACM 21(2), 135–139 (1978)
Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Mathematical Programming 69(1), 111–147 (1995)
Lengauer, T.: Combinatorial Algorithms for Integrated Circuit Layout. John Wiley & Sons, Inc., New York, NY, USA (1990)
Liu, J.W.H.: Modification of the minimum-degree algorithm by multiple elimination. ACM Trans. Math. Softw. 11(2), 141–153 (1985)
Mattson, T., Bader, D.A., Berry, J.W., Buluç, A., Dongarra, J.J., Faloutsos, C., Feo, J., Gilbert, J.R., Gonzalez, J., Hendrickson, B., Kepner, J., Leiserson, C.E., Lumsdaine, A., Padua, D.A., Poole, S.W., Reinhardt, S.P., Stonebraker, M., Wallach, S., Yoo, A.: Standards for graph algorithm primitives. CoRR abs/1408.0393 (2014)
Medhi, D.: Parallel bundle-based decomposition for large-scale structured mathematical programming problems. Annals of Operations Research 22(1), 101–127 (1990)
Medhi, D.: Bundle-based decomposition for large-scale convex optimization: Error estimate and application to block-angular linear programs. Mathematical Programming 66(1), 79–101 (1994)
Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM Journal on Optimization 2(4), 575–601 (1992)
Naumov, M., Manguoglu, M., Sameh, A.H.: A tearing-based hybrid parallel sparse linear system solver. J. Comput. Appl. Math. 234(10), 3025–3038 (2010)
Naumov, M., Sameh, A.H.: A tearing-based hybrid parallel banded linear system solver. J. Comput. Appl. Math. 226(2), 306–318 (2009)
Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)
Pellegrini, F., Roman, J.: Scotch: A software package for static mapping by dual recursive bipartitioning of process and architecture graphs. In: H. Liddell, A. Colbrook, B. Hertzberger, P. Sloot (eds.) High-Performance Computing and Networking, Lecture Notes in Computer Science, vol. 1067, pp. 493–498. Springer Berlin Heidelberg (1996)
Pinar, A., Çatalyürek, Ü.V., Aykanat, C., Pinar, M.: Decomposing linear programs for parallel solution. In: J. Dongarra, K. Madsen, J. Waśniewski (eds.) Applied Parallel Computing Computations in Physics, Chemistry and Engineering Science, pp. 473–482. Springer Berlin Heidelberg, Berlin, Heidelberg (1996)
Pothen, A., Simon, H.D., Liou, K.P.: Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl. 11(3), 430–452 (1990)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (2003)
Sameh, A.H., Sarin, V.: Hybrid parallel linear system solvers. Int. J. Comp. Fluid Dyn 12, 213–223 (1998)
Selvitopi, O., Aykanat, C.: Reducing latency cost in 2d sparse matrix partitioning models. Parallel Computing 57, 1–24 (2016)
Tinney, W.F., Walker, J.W.: Direct solutions of sparse network equations by optimally ordered triangular factorization. Proceedings of the IEEE 55(11), 1801–1809 (1967)
Torun, F.S., Manguoglu, M., Aykanat, C.: Parallel minimum norm solution of sparse block diagonal column overlapped underdetermined systems. ACM Trans. Math. Softw. 43(4), 31:1–31:21 (2017)
Uçar, B., Aykanat, C.: Minimizing communication cost in fine-grain partitioning of sparse matrices. In: A. Yazıcı, C. Şener (eds.) Computer and Information Sciences - ISCIS 2003, Lecture Notes in Computer Science, vol. 2869, pp. 926–933. Springer Berlin Heidelberg (2003)
Uçar, B., Aykanat, C.: Encapsulating multiple communication-cost metrics in partitioning sparse rectangular matrices for parallel matrix-vector multiplies. SIAM J. Sci. Comput. 25(6), 1837–1859 (2004)
Uçar, B., Aykanat, C., Pinar, M.c., Malas, T.: Parallel image restoration using surrogate constraint methods. J. Parallel Distrib. Comput. 67(2), 186–204 (2007)
Yang, K., Murty, K.G.: New iterative methods for linear inequalities. Journal of Optimization Theory and Applications 72(1), 163–185 (1992)
Acknowledgements
This work was supported by the Director, Office of Science, U.S. Department of Energy under Contract No. DE-AC02-05CH11231. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Selvitopi, O., Acer, S., Manguoğlu, M., Aykanat, C. (2020). The Effect of Various Sparsity Structures on Parallelism and Algorithms to Reveal Those Structures. In: Grama, A., Sameh, A. (eds) Parallel Algorithms in Computational Science and Engineering. Modeling and Simulation in Science, Engineering and Technology. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-43736-7_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-43736-7_2
Published:
Publisher Name: Birkhäuser, Cham
Print ISBN: 978-3-030-43735-0
Online ISBN: 978-3-030-43736-7
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)