Skip to main content
View expanded cover

Parallel Algorithms in Computational Science and Engineering pp 35–62Cite as

Birkhäuser

The Effect of Various Sparsity Structures on Parallelism and Algorithms to Reveal Those Structures

  • 747 Accesses

Part of the Modeling and Simulation in Science, Engineering and Technology book series (MSSET)

Abstract

Structured sparse matrices can greatly benefit parallel numerical methods in terms of parallel performance and convergence. In this chapter, we present combinatorial models for obtaining several different sparse matrix forms. There are four basic forms we focus on: singly-bordered block-diagonal form, doubly-bordered block-diagonal form, nonempty off-diagonal block minimization, and block diagonal with overlap form. For each of these forms, we first present the form in detail and describe what goals are sought within the form, and then examine the combinatorial models that attain the respective form while targeting the sought goals, and finally explain in which aspects the forms benefit certain parallel numerical methods and their relationship with the models. Our work focuses especially on graph and hypergraph partitioning models in obtaining the mentioned forms. Despite their relatively high preprocessing overhead compared to other heuristics, they have proven to model the given problem more accurately and this overhead can be often amortized due the fact that matrix structure does not change much during a typical numerical simulation. This chapter presents a number of models and their relationship with parallel numerical methods.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-43736-7_2
  • Chapter length: 28 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   34.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-43736-7
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   44.99
Price excludes VAT (USA)
Hardcover Book
USD   59.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  1. Acer, S., Aykanat, C.: Reordering sparse matrices into block-diagonal column-overlapped form. Journal of Parallel and Distributed Computing 140, 99–109 (2020)

    Google Scholar 

  2. Acer, S., Kayaaslan, E., Aykanat, C.: A recursive bipartitioning algorithm for permuting sparse square matrices into block diagonal form with overlap. SIAM Journal on Scientific Computing 35(1), C99–C121 (2013)

    MathSciNet  MATH  Google Scholar 

  3. Alpert, C.J., Kahng, A.B.: Recent directions in netlist partitioning: a survey. Integr. VLSI J. 19, 1–81 (1995)

    MATH  Google Scholar 

  4. Amestoy, P., Davis, T., Duff, I.: An approximate minimum degree ordering algorithm. SIAM Journal on Matrix Analysis and Applications 17(4), 886–905 (1996)

    MathSciNet  MATH  Google Scholar 

  5. Aykanat, C., Pinar, A., Çatalyürek, U.V.: Permuting sparse rectangular matrices into block-diagonal form. SIAM J. Sci. Comput. 25, 1860–1879 (2004)

    MathSciNet  MATH  Google Scholar 

  6. Bjorck, A.: Numerical Methods for Least Squares Problems. Society for Industrial and Applied Mathematics (1996)

    Google Scholar 

  7. Brandstädt, A., Le, V.B., Spinrad, J.P.: Graph Classes: A Survey. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (1999)

    Google Scholar 

  8. Bui, T.N., Jones, C.: Finding good approximate vertex and edge partitions is np-hard. Inf. Process. Lett. 42(3), 153–159 (1992)

    MathSciNet  MATH  Google Scholar 

  9. Bui, T.N., Jones, C.: A heuristic for reducing fill-in in sparse matrix factorization. In: Proceedings of the 6th SIAM Conference on Parallel Processing for Scientific Computing, pp. 445–452. Society for Industrial and Applied Mathematics (1993)

    Google Scholar 

  10. Catalyurek, U., Aykanat, C.: Hypergraph-partitioning-based decomposition for parallel sparse-matrix vector multiplication. IEEE Trans. Parallel Distrib. Syst. 10, 673–693 (1999)

    Google Scholar 

  11. Catalyurek, U., Aykanat, C., Kayaaslan, E.: Hypergraph partitioning-based fill-reducing ordering for symmetric matrices. SIAM Journal on Scientific Computing 33(4), 1996–2023 (2011)

    MathSciNet  MATH  Google Scholar 

  12. Çatalyürek, U.V., Aykanat, C.: Patoh: partitioning tool for hypergraphs. Tech. rep., Department of Computer Engineering, Bilkent University (1999)

    Google Scholar 

  13. Cong, J., Lubio, W., Shivakumur, N.: Multi-way VLSI circuit partitioning based on dual net representation. In: IEEE/ACM International Conference on Computer-Aided Design, pp. 56–62 (1994)

    Google Scholar 

  14. Dantzig, G.B., Wolfe, P.: Decomposition principle for linear programs. Oper. Res. 8(1), 101–111 (1960)

    MATH  Google Scholar 

  15. Ferris, M.C., Horn, J.D.: Partitioning mathematical programs for parallel solution. Mathematical Programming 80(1), 35–61 (1998)

    MathSciNet  MATH  Google Scholar 

  16. Freund, R., Nachtigal, N.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numerische Mathematik 60(1), 315–339 (1991)

    MathSciNet  MATH  Google Scholar 

  17. George, A.: Nested dissection of a regular finite element mesh. SIAM Journal on Numerical Analysis 10(2), 345–363 (1973)

    MathSciNet  MATH  Google Scholar 

  18. George, A., Liu, J.W.: Computer Solution of Large Sparse Positive Definite. Prentice Hall Professional Technical Reference (1981)

    Google Scholar 

  19. Golub, G., Sameh, A.H., Sarin, V.: A parallel balance scheme for banded linear systems. Numerical Linear Algebra with Applications 8(5), 285–299 (2001)

    MathSciNet  MATH  Google Scholar 

  20. Hendrickson, B., Leland, R.: The Chaco user’s guide version 2.0 (1995)

    Google Scholar 

  21. Hendrickson, B., Leland, R.: A multilevel algorithm for partitioning graphs. In: Proceedings of the 1995 ACM/IEEE conference on Supercomputing (CDROM), Supercomputing ’95. ACM, New York, NY, USA (1995)

    Google Scholar 

  22. Hendrickson, B., Rothberg, E.: Improving the run time and quality of nested dissection ordering. SIAM J. Sci. Comput. 20(2), 468–489 (1998)

    MathSciNet  MATH  Google Scholar 

  23. Kahou, G.A.A., Grigori, L., Sosonkina, M.: A partitioning algorithm for block-diagonal matrices with overlap. Parallel Comput. 34(6–8), 332–344 (2008)

    MathSciNet  Google Scholar 

  24. Kahou, G.A.A., Kamgnia, E., Philippe, B.: An explicit formulation of the multiplicative Schwarz preconditioner. Applied Numerical Mathematics 57(11), 1197–1213 (2007). Numerical Algorithms, Parallelism and Applications (2)

    Google Scholar 

  25. Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)

    MathSciNet  MATH  Google Scholar 

  26. Karsavuran, M.O., Akbudak, K., Aykanat, C.: Locality-aware parallel sparse matrix-vector and matrix-transpose-vector multiplication on many-core processors. IEEE Transactions on Parallel and Distributed Systems 27(6), 1713–1726 (2016)

    Google Scholar 

  27. Karypis, G., Aggarwal, R., Kumar, V., Shekhar, S.: Multilevel hypergraph partitioning: applications in VLSI domain. IEEE Trans. Very Large Scale Integr. Syst. 7, 69–79 (1999)

    Google Scholar 

  28. Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359–392 (1998)

    MathSciNet  MATH  Google Scholar 

  29. Kayaaslan, E., Pinar, A., Çatalyürek, U., Aykanat, C.: Partitioning hypergraphs in scientific computing applications through vertex separators on graphs. SIAM J. Sci. Comput. 34(2), 970–992 (2012)

    MathSciNet  MATH  Google Scholar 

  30. Kou, L.T., Stockmeyer, L.J., Wong, C.K.: Covering edges by cliques with regard to keyword conflicts and intersection graphs. Commun. ACM 21(2), 135–139 (1978)

    MathSciNet  MATH  Google Scholar 

  31. Lemaréchal, C., Nemirovskii, A., Nesterov, Y.: New variants of bundle methods. Mathematical Programming 69(1), 111–147 (1995)

    MathSciNet  MATH  Google Scholar 

  32. Lengauer, T.: Combinatorial Algorithms for Integrated Circuit Layout. John Wiley & Sons, Inc., New York, NY, USA (1990)

    MATH  Google Scholar 

  33. Liu, J.W.H.: Modification of the minimum-degree algorithm by multiple elimination. ACM Trans. Math. Softw. 11(2), 141–153 (1985)

    MathSciNet  MATH  Google Scholar 

  34. Mattson, T., Bader, D.A., Berry, J.W., Buluç, A., Dongarra, J.J., Faloutsos, C., Feo, J., Gilbert, J.R., Gonzalez, J., Hendrickson, B., Kepner, J., Leiserson, C.E., Lumsdaine, A., Padua, D.A., Poole, S.W., Reinhardt, S.P., Stonebraker, M., Wallach, S., Yoo, A.: Standards for graph algorithm primitives. CoRR abs/1408.0393 (2014)

    Google Scholar 

  35. Medhi, D.: Parallel bundle-based decomposition for large-scale structured mathematical programming problems. Annals of Operations Research 22(1), 101–127 (1990)

    MathSciNet  MATH  Google Scholar 

  36. Medhi, D.: Bundle-based decomposition for large-scale convex optimization: Error estimate and application to block-angular linear programs. Mathematical Programming 66(1), 79–101 (1994)

    MathSciNet  MATH  Google Scholar 

  37. Mehrotra, S.: On the implementation of a primal-dual interior point method. SIAM Journal on Optimization 2(4), 575–601 (1992)

    MathSciNet  MATH  Google Scholar 

  38. Naumov, M., Manguoglu, M., Sameh, A.H.: A tearing-based hybrid parallel sparse linear system solver. J. Comput. Appl. Math. 234(10), 3025–3038 (2010)

    MathSciNet  MATH  Google Scholar 

  39. Naumov, M., Sameh, A.H.: A tearing-based hybrid parallel banded linear system solver. J. Comput. Appl. Math. 226(2), 306–318 (2009)

    MathSciNet  MATH  Google Scholar 

  40. Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)

    MathSciNet  MATH  Google Scholar 

  41. Pellegrini, F., Roman, J.: Scotch: A software package for static mapping by dual recursive bipartitioning of process and architecture graphs. In: H. Liddell, A. Colbrook, B. Hertzberger, P. Sloot (eds.) High-Performance Computing and Networking, Lecture Notes in Computer Science, vol. 1067, pp. 493–498. Springer Berlin Heidelberg (1996)

    Google Scholar 

  42. Pinar, A., Çatalyürek, Ü.V., Aykanat, C., Pinar, M.: Decomposing linear programs for parallel solution. In: J. Dongarra, K. Madsen, J. Waśniewski (eds.) Applied Parallel Computing Computations in Physics, Chemistry and Engineering Science, pp. 473–482. Springer Berlin Heidelberg, Berlin, Heidelberg (1996)

    Google Scholar 

  43. Pothen, A., Simon, H.D., Liou, K.P.: Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl. 11(3), 430–452 (1990)

    MathSciNet  MATH  Google Scholar 

  44. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA (2003)

    Google Scholar 

  45. Sameh, A.H., Sarin, V.: Hybrid parallel linear system solvers. Int. J. Comp. Fluid Dyn 12, 213–223 (1998)

    MathSciNet  MATH  Google Scholar 

  46. Selvitopi, O., Aykanat, C.: Reducing latency cost in 2d sparse matrix partitioning models. Parallel Computing 57, 1–24 (2016)

    MathSciNet  Google Scholar 

  47. Tinney, W.F., Walker, J.W.: Direct solutions of sparse network equations by optimally ordered triangular factorization. Proceedings of the IEEE 55(11), 1801–1809 (1967)

    Google Scholar 

  48. Torun, F.S., Manguoglu, M., Aykanat, C.: Parallel minimum norm solution of sparse block diagonal column overlapped underdetermined systems. ACM Trans. Math. Softw. 43(4), 31:1–31:21 (2017)

    Google Scholar 

  49. Uçar, B., Aykanat, C.: Minimizing communication cost in fine-grain partitioning of sparse matrices. In: A. Yazıcı, C. Şener (eds.) Computer and Information Sciences - ISCIS 2003, Lecture Notes in Computer Science, vol. 2869, pp. 926–933. Springer Berlin Heidelberg (2003)

    Google Scholar 

  50. Uçar, B., Aykanat, C.: Encapsulating multiple communication-cost metrics in partitioning sparse rectangular matrices for parallel matrix-vector multiplies. SIAM J. Sci. Comput. 25(6), 1837–1859 (2004)

    MathSciNet  MATH  Google Scholar 

  51. Uçar, B., Aykanat, C., Pinar, M.c., Malas, T.: Parallel image restoration using surrogate constraint methods. J. Parallel Distrib. Comput. 67(2), 186–204 (2007)

    Google Scholar 

  52. Yang, K., Murty, K.G.: New iterative methods for linear inequalities. Journal of Optimization Theory and Applications 72(1), 163–185 (1992)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the Director, Office of Science, U.S. Department of Energy under Contract No. DE-AC02-05CH11231. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia LLC., a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA-0003525.

Author information

Authors and Affiliations

  1. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Oguz Selvitopi

  2. Center for Computing Research, Sandia National Laboratories, Albuquerque, NM, USA

    Seher Acer

  3. Department of Computer Engineering, Middle East Technical University, Ankara, Turkey

    Murat Manguoğlu

  4. Department of Computer Engineering, Bilkent University, Ankara, Turkey

    Cevdet Aykanat

Authors
  1. Oguz Selvitopi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seher Acer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Murat Manguoğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Cevdet Aykanat
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Murat Manguoğlu .

Editor information

Editors and Affiliations

  1. Purdue University, West Lafayette, IN, USA

    Prof. Ananth Grama

  2. Purdue University, West Lafayette, IN, USA

    Prof. Ahmed H. Sameh

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Selvitopi, O., Acer, S., Manguoğlu, M., Aykanat, C. (2020). The Effect of Various Sparsity Structures on Parallelism and Algorithms to Reveal Those Structures. In: Grama, A., Sameh, A. (eds) Parallel Algorithms in Computational Science and Engineering. Modeling and Simulation in Science, Engineering and Technology. Birkhäuser, Cham. https://doi.org/10.1007/978-3-030-43736-7_2

Download citation