Banded Linear Systems

  • Efstratios Gallopoulos
  • Bernard Philippe
  • Ahmed H. Sameh
Part of the Scientific Computation book series (SCIENTCOMP)


We encounter banded linear systems in many areas of computational science and engineering, including computational mechanics and nanoelectronics, to name but a few.


  1. 1.
    Arbenz, P., Hegland, M.: On the stable parallel solution of general narrow banded linear systems. High Perform. Algorithms Struct. Matrix Probl. 47–73 (1998)Google Scholar
  2. 2.
    Arbenz, P., Cleary, A., Dongarra, J., Hegland, M.: A comparison of parallel solvers for general narrow banded linear systems. Parallel Distrib. Comput. Pract. 2(4), 385–400 (1999)MATHGoogle Scholar
  3. 3.
    Blackford, L., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.: ScaLAPACK User’s Guide. SIAM, Philadelphia (1997). URL
  4. 4.
    Conroy, J.: Parallel algorithms for the solution of narrow banded systems. Appl. Numer. Math. 5, 409–421 (1989)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Dongarra, J., Johnsson, L.: Solving banded systems on a parallel processor. Parallel Comput. 5(1–2), 219–246 (1987)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    George, A.: Numerical experiments using dissection methods to solve \(n\) by \(n\) grid problems. SIAM J. Numer. Anal. 14, 161–179 (1977)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Golub, G., Sameh, A., Sarin, V.: A parallel balance scheme for banded linear systems. Numer. Linear Algebra Appl. 8, 297–316 (2001)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Johnsson, S.: Solving narrow banded systems on ensemble architectures. ACM Trans. Math. Softw. 11, 271–288 (1985)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Meier, U.: A parallel partition method for solving banded systems of linear equations. Parallel Comput. 2, 33–43 (1985)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Tang, W.: Generalized Schwarz splittings. SIAM J. Sci. Stat. Comput. 13, 573–595 (1992)CrossRefMATHGoogle Scholar
  11. 11.
    Wright, S.: Parallel algorithms for banded linear systems. SIAM J. Sci. Stat. Comput. 12, 824–842 (1991)CrossRefMATHGoogle Scholar
  12. 12.
    Sameh, A., Kuck, D.: On stable parallel linear system solvers. J. Assoc. Comput. Mach. 25(1), 81–91 (1978)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Dongarra, J.J., Sameh, A.: On some parallel banded system solvers. Technical Report ANL/MCS-TM-27, Mathematics Computer Science Division at Argonne National Laboratory (1984)Google Scholar
  14. 14.
    Gallivan, K., Gallopoulos, E., Sameh, A.: CEDAR—an experiment in parallel computing. Comput. Math. Appl. 1(1), 77–98 (1994)MATHGoogle Scholar
  15. 15.
    Lawrie, D.H., Sameh, A.: The computation and communication complexity of a parallel banded system solver. ACM TOMS 10(2), 185–195 (1984)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Polizzi, E., Sameh, A.: A parallel hybrid banded system solver: the SPIKE algorithm. Parallel Comput. 32, 177–194 (2006)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Polizzi, E., Sameh, A.: SPIKE: a parallel environment for solving banded linear systems. Compon. Fluids 36, 113–120 (2007)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Sameh, A., Kuck, D.: A parallel QR algorithm for symmetric tridiagonal matrices. IEEE Trans. Comput. 26(2), 147–153 (1977)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Anderson, E., Bai, Z., Bischof, C., Blackford, S., Demmel, J., Dongarra, J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., Sorensen, D.: LAPACK Users’ Guide, 3rd edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)CrossRefMATHGoogle Scholar
  20. 20.
    Demko, S., Moss, W., Smith, P.: Decay rates for inverses of band matrices. Math. Comput. 43(168), 491–499 (1984)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefMATHGoogle Scholar
  22. 22.
    Golub, G., Van Loan, C.: Matrix Computations, 4th edn. Johns Hopkins. University Press, Baltimore (2013)Google Scholar
  23. 23.
    Davis, T.: Algorithm 915, SuiteSparseQR: multifrontal multithreaded rank-revealing sparse QR factorization. ACM Trans. Math. Softw. 38(1), 8:1–8:22 (2011). doi: 10.1145/2049662.2049670, URL
  24. 24.
    Lou, G.: Parallel methods for solving linear systems via overlapping decompositions. Ph.D. thesis, University of Illinois at Urbana-Champaign (1989)Google Scholar
  25. 25.
    Naumov, M., Sameh, A.: A tearing-based hybrid parallel banded linear system solver. J. Comput. Appl. Math. 226, 306–318 (2009)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Benzi, M., Golub, G., Liesen, J.: Numerical solution of saddle-point problems. Acta Numer. 1–137 (2005)Google Scholar
  27. 27.
    Hockney, R., Jesshope, C.: Parallel Computers. Adam Hilger (1983)Google Scholar
  28. 28.
    Ortega, J.M.: Introduction to Parallel and Vector Solution of Linear Systems. Plenum Press, New York (1988)CrossRefMATHGoogle Scholar
  29. 29.
    Golub, G., Ortega, J.: Scientific Computing: An Introduction with Parallel Computing. Academic Press Inc., San Diego (1993)Google Scholar
  30. 30.
    Davidson, A., Zhang, Y., Owens, J.: An auto-tuned method for solving large tridiagonal systems on the GPU. In: Proceedings of IEEE IPDPS, pp. 956–965 (2011)Google Scholar
  31. 31.
    Lopez, J., Zapata, E.: Unified architecture for divide and conquer based tridiagonal system solvers. IEEE Trans. Comput. 43(12), 1413–1425 (1994). doi: 10.1109/12.338101 CrossRefGoogle Scholar
  32. 32.
    Santos, E.: Optimal and efficient parallel tridiagonal solvers using direct methods. J. Supercomput. 30(2), 97–115 (2004). doi: 10.1023/B:SUPE.0000040615.60545.c6, URL
  33. 33.
    Chang, L.W., Stratton, J., Kim, H., Hwu, W.M.: A scalable, numerically stable, high-performance tridiagonal solver using GPUs. In: Proceedings International Conference High Performance Computing, Networking Storage and Analysis, SC’12, pp. 27:1–27:11. IEEE Computer Society Press, Los Alamitos (2012). URL
  34. 34.
    Goeddeke, D., Strzodka, R.: Cyclic reduction tridiagonal solvers on GPUs applied to mixed-precision multigrid. IEEE Trans. Parallel Distrib. Syst. 22(1), 22–32 (2011)CrossRefGoogle Scholar
  35. 35.
    Codenotti, B., Leoncini, M.: Parallel Complexity of Linear System Solution. World Scientific, Singapore (1991)CrossRefGoogle Scholar
  36. 36.
    Ascher, U., Mattheij, R., Russell, R.: Numerical Solution of Boundary Value Problems for Ordinary Differential Equations. Classics in Applied Mathematics. SIAM, Philadelphia (1995)CrossRefMATHGoogle Scholar
  37. 37.
    Isaacson, E., Keller, H.B.: Analysis of Numerical Methods. Wiley, New York (1966)MATHGoogle Scholar
  38. 38.
    Keller, H.B.: Numerical Methods for Two-Point Boundary-Value Problems. Dover Publications, New York (1992)Google Scholar
  39. 39.
    Bank, R.E.: Marching algorithms and block Gaussian elimination. In: Bunch, J.R., Rose, D. (eds.) Sparse Matrix Computations, pp. 293–307. Academic Press, New York (1976)Google Scholar
  40. 40.
    Bank, R.E., Rose, D.: Marching algorithms for elliptic boundary value problems. I: the constant coefficient case. SIAM J. Numer. Anal. 14(5), 792–829 (1977)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Roache, P.: Elliptic Marching Methods and Domain Decomposition. CRC Press Inc., Boca Raton (1995)Google Scholar
  42. 42.
    Richardson, L.F.: Weather Prediction by Numerical Process. Cambridge University Press. Reprinted by Dover Publications, 1965 (1922)Google Scholar
  43. 43.
    Arbenz, P., Hegland, M.: The stable parallel solution of narrow banded linear systems. In: Heath, M., et al. (eds.) Proceedings of Eighth SIAM Conference Parallel Processing and Scientific Computing SIAM, Philadelphia (1997)Google Scholar
  44. 44.
    Bank, R.E., Rose, D.: Marching algorithms for elliptic boundary value problems. II: the variable coefficient case. SIAM J. Numer. Anal. 14(5), 950–969 (1977)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Higham, N.: Accuracy and Stability of Numerical Algorithms, 2nd edn. SIAM, Philadelphia (2002)CrossRefMATHGoogle Scholar
  46. 46.
    Higham, N.: Stability of parallel triangular system solvers. SIAM J. Sci. Comput. 16(2), 400–413 (1995)MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Viswanath, D., Trefethen, L.: Condition numbers of random triangular matrices. SIAM J. Matrix Anal. Appl. 19(2), 564–581 (1998)MathSciNetCrossRefMATHGoogle Scholar
  48. 48.
    Hockney, R.: A fast direct solution of Poisson’s equation using Fourier analysis. J. Assoc. Comput. Mach. 12, 95–113 (1965)MathSciNetCrossRefMATHGoogle Scholar
  49. 49.
    Gander, W., Golub, G.H.: Cyclic reduction: history and applications. In: Luk, F., Plemmons, R. (eds.) Proceedings of the Workshop on Scientific Computing, pp. 73–85. Springer, New York (1997). URL
  50. 50.
    Amodio, P., Brugnano, L.: Parallel factorizations and parallel solvers for tridiagonal linear systems. Linear Algebra Appl. 172, 347–364 (1992). doi: 10.1016/0024-3795(92)90034-8, URL
  51. 51.
    Heller, D.: Some aspects of the cyclic reduction algorithm for block tridiagonal linear systems. SIAM J. Numer. Anal. 13(4), 484–496 (1976)MathSciNetCrossRefMATHGoogle Scholar
  52. 52.
    Lambiotte Jr, J., Voigt, R.: The solution of tridiagonal linear systems on the CDC STAR 100 computer. ACM Trans. Math. Softw. 1(4), 308–329 (1975). doi: 10.1145/355656.355658, URL
  53. 53.
    Nassimi, D., Sahni, S.: An optimal routing algorithm for mesh-connected parallel computers. J. Assoc. Comput. Mach. 27(1), 6–29 (1980)MathSciNetCrossRefMATHGoogle Scholar
  54. 54.
    Nassimi, D., Sahni, S.: Parallel permutation and sorting algorithms and a new generalized connection network. J. Assoc. Comput. Mach. 29(3), 642–667 (1982)MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    George, A.: Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10(2), 345–363 (1973). URL
  56. 56.
    Amodio, P., Brugnano, L., Politi, T.: Parallel factorization for tridiagonal matrices. SIAM J. Numer. Anal. 30(3), 813–823 (1993)MathSciNetCrossRefMATHGoogle Scholar
  57. 57.
    Johnsson, S.: Solving tridiagonal systems on ensemble architectures. SIAM J. Sci. Stat. Comput. 8, 354–392 (1987)MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    Zhang, Y., Cohen, J., Owens, J.: Fast tridiagonal solvers on the GPU. ACM SIGPLAN Not. 45(5), 127–136 (2010)CrossRefGoogle Scholar
  59. 59.
    Amodio, P., Mazzia, F.: Backward error analysis of cyclic reduction for the solution of tridiagonal systems. Math. Comput. 62(206), 601–617 (1994)MathSciNetCrossRefMATHGoogle Scholar
  60. 60.
    Higham, N.: Bounding the error in Gaussian elimination for tridiagonal systems. SIAM J. Matrix Anal. Appl. 11(4), 521–530 (1990)MathSciNetCrossRefMATHGoogle Scholar
  61. 61.
    Zhang, Y., Owens, J.: A quantitative performance analysis model for GPU architectures. In: Proceedings of the 17th IEEE International Symposium on High-Performance Computer Architecture (HPCA 17) (2011)Google Scholar
  62. 62.
    El-Mikkawy, M., Sogabe, T.: A new family of k-Fibonacci numbers. Appl. Math. Comput. 215(12), 4456–4461 (2010). URL
  63. 63.
    Fang, H.R., O’Leary, D.: Stable factorizations of symmetric tridiagonal and triadic matrices. SIAM J. Math. Anal. Appl. 28(2), 576–595 (2006)MathSciNetCrossRefGoogle Scholar
  64. 64.
    Mikkelsen, C., Kågström, B.: Parallel solution of narrow banded diagonally dominant linear systems. In: Jónasson, L. (ed.) PARA 2010. LNCS, vol. 7134, pp. 280–290. Springer (2012). doi: 10.1007/978-3-642-28145-7_28, URL
  65. 65.
    Mikkelsen, C., Kågström, B.: Approximate incomplete cyclic reduction for systems which are tridiagonal and strictly diagonally dominant by rows. In: Manninen, P., Öster, P. (eds.) PARA 2012. LNCS, vol. 7782, pp. 250–264. Springer (2013). doi: 10.1007/978-3-642-36803-5_18, URL
  66. 66.
    Bini, D., Meini, B.: The cyclic reduction algorithm: from Poisson equation to stochastic processes and beyond. Numer. Algorithms 51(1), 23–60 (2008). doi: 10.1007/s11075-008-9253-0, URL;
  67. 67.
    Sameh, A.: Numerical parallel algorithms—a survey. In: Kuck, D., Lawrie, D., Sameh, A. (eds.) High Speed Computer and Algorithm Optimization, pp. 207–228. Academic Press, Sans Diego (1977)Google Scholar
  68. 68.
    Mathias, R.: The instability of parallel prefix matrix multiplication. SIAM J. Sci. Comput. 16(4) (1995), to appearGoogle Scholar
  69. 69.
    Eğecioğlu, O., Koç, C., Laub, A.: A recursive doubling algorithm for solution of tridiagonal systems on hypercube multiprocessors. J. Comput. Appl. Math. 27, 95–108 (1989)MathSciNetCrossRefMATHGoogle Scholar
  70. 70.
    Dubois, P., Rodrigue, G.: An analysis of the recursive doubling algorithm. In: Kuck, D., Lawrie, D., Sameh, A. (eds.) High Speed Computer and Algorithm Organization, pp. 299–305. Academic Press, San Diego (1977)Google Scholar
  71. 71.
    Hammarling, S.: A survey of numerical aspects of plane rotations. Report Maths. 1, Middlesex Polytechnic (1977). URL Available as Manchester Institute for Mathematical Sciences MIMS EPrint 2008.69
  72. 72.
    Bar-On, I., Codenotti, B.: A fast and stable parallel QR algorithm for symmetric tridiagonal matrices. Linear Algebra Appl. 220, 63–95 (1995). doi: 10.1016/0024-3795(93)00360-C, URL
  73. 73.
    Gill, P.E., Golub, G., Murray, W., Saunders, M.: Methods for modifying matrix factorizations. Math. Comput. 28, 505–535 (1974)MathSciNetCrossRefMATHGoogle Scholar
  74. 74.
    Lakshmivarahan, S., Dhall, S.: Parallelism in the Prefix Problem. Oxford University Press, New York (1994)Google Scholar
  75. 75.
    Cleary, A., Dongarra, J.: Implementation in ScaLAPACK of divide and conquer algorithms for banded and tridiagonal linear systems. Technical Report UT-CS-97-358, University of Tennessee Computer Science Technical Report (1997)Google Scholar
  76. 76.
    Bar-On, I., Codenotti, B., Leoncini, M.: Checking robust nonsingularity of tridiagonal matrices in linear time. BIT Numer. Math. 36(2), 206–220 (1996). doi: 10.1007/BF01731979, URL
  77. 77.
    Bar-On, I.: Checking non-singularity of tridiagonal matrices. Electron. J. Linear Algebra 6, 11–19 (1999). URL
  78. 78.
    Bondeli, S.: Divide and conquer: a parallel algorithm for the solution of a tridiagonal system of equations. Parallel Comput. 17, 419–434 (1991)CrossRefMATHGoogle Scholar
  79. 79.
    Wang, H.: A parallel method for tridiagonal equations. ACM Trans. Math. Softw. 7, 170–183 (1981)CrossRefMATHGoogle Scholar
  80. 80.
    Wright, S.: Parallel algorithms for banded linear systems. SIAM J. Sci. Stat. Comput. 12(4), 824–842 (1991)CrossRefMATHGoogle Scholar
  81. 81.
    Stewart, G.: Modifying pivot elements in Gaussian elimination. Math. Comput. 28(126), 537–542 (1974)CrossRefMATHGoogle Scholar
  82. 82.
    Li, X., Demmel, J.: SuperLU-DIST: A scalable distributed-memory sparse direct solver for unsymmetric linear systems. ACM TOMS 29(2), 110–140 (2003). URL
  83. 83.
    Venetis, I.E., Kouris, A., Sobczyk, A., Gallopoulos, E., Sameh, A.: A direct tridiagonal solver based on Givens rotations for GPU-based architectures. Technical Report HPCLAB-SCG-06/11-14, CEID, University of Patras (2014)Google Scholar
  84. 84.
    Bunch, J.: Partial pivoting strategies for symmetric matrices. SIAM J. Numer. Anal. 11(3), 521–528 (1974)MathSciNetCrossRefMATHGoogle Scholar
  85. 85.
    Bunch, J., Kaufman, K.: Some stable methods for calculating inertia and solving symmetric linear systems. Math. Comput. 31, 162–179 (1977)MathSciNetCrossRefGoogle Scholar
  86. 86.
    Erway, J., Marcia, R.: A backward stability analysis of diagonal pivoting methods for solving unsymmetric tridiagonal systems without interchanges. Numer. Linear Algebra Appl. 18, 41–54 (2011). doi: 10.1002/nla.674, URL
  87. 87.
    Erway, J.B., Marcia, R.F., Tyson, J.: Generalized diagonal pivoting methods for tridiagonal systems without interchanges. IAENG Int. J. Appl. Math. 4(40), 269–275 (2010)MathSciNetGoogle Scholar
  88. 88.
    Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2009)CrossRefGoogle Scholar
  89. 89.
    Vandebril, R., Van Barel, M., Mastronardi, N.: Matrix Computations and Semiseparable Matrices. Volume I: Linear Systems. Johns Hopkins University Press (2008)Google Scholar
  90. 90.
    Gantmacher, F., Krein, M.: Sur les matrices oscillatoires et complèments non négatives. Composition Mathematica 4, 445–476 (1937)Google Scholar
  91. 91.
    Bukhberger, B., Emelyneko, G.: Methods of inverting tridiagonal matrices. USSR Comput. Math. Math. Phys. 13, 10–20 (1973)CrossRefGoogle Scholar
  92. 92.
    Swarztrauber, P.N.: A parallel algorithm for solving general tridiagonal equations. Math. Comput. 33, 185–199 (1979)MathSciNetCrossRefMATHGoogle Scholar
  93. 93.
    Yamamoto, T., Ikebe, Y.: Inversion of band matrices. Linear Algebra Appl. 24, 105–111 (1979). doi: 10.1016/0024-3795(79)90151-4, URL
  94. 94.
    Strang, G., Nguyen, T.: The interplay of ranks of submatrices. SIAM Rev. 46(4), 637–646 (2004). URL

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Efstratios Gallopoulos
    • 1
  • Bernard Philippe
    • 2
  • Ahmed H. Sameh
    • 3
  1. 1.Computer Engineering and Informatics DepartmentUniversity of PatrasPatrasGreece
  2. 2.Campus de BeaulieuINRIA/IRISARennes CedexFrance
  3. 3.Department of Computer SciencePurdue UniversityWest LafayetteUSA

Personalised recommendations