Skip to main content
SpringerLink
Log in

A breakdown-free block conjugate gradient method

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

In this paper, we analyze all possible situations of rank deficiency that cause breakdown in block conjugate gradient (BCG) solvers. A simple solution, breakdown-free block conjugate gradient (BFBCG), is designed to address the rank deficiency problem. The rationale of the BFBCG algorithm is to derive new forms of parameter matrices based on the potentially reduced search subspace to handle rank deficiency. Orthogonality properties and convergence of BFBCG in case of rank deficiency are justified accordingly with mathematical rigor. BFBCG yields faster convergence than restarting BCG when breakdown occurs. Numerical examples suffering from rank deficiency are provided to demonstrate the robustness of BFBCG.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Agullo, E., Giraud, L., Jing, Y.F.: Block GMRES method with inexact breakdowns and deflated restarting. SIAM J. Matrix Anal. Appl. 35(4), 1625–1651 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  2. Axelsson, O.: On preconditioning and convergence acceleration in sparse matrix problems. CERN 74–10, Genèva, Switzerland (1974)

  3. Barrett, R., Berry, M., Chan, T., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., van der Vorst, H.: Templates for the solution of linear systems: building blocks for iterative methods. SIAM, Philadelphia (1994)

    Book  MATH  Google Scholar 

  4. Birk, S., Frommer, A.: A CG method for multiple right hand sides and multiple shifts in lattice QCD calculations. In: Proceedings of the XXIX International Symposium on Lattice Field Theory, pp. 10–16 (2011)

  5. Broyden, C.G.: A breakdown of the block CG method. Optim. Method. Softw. 7(1), 41–55 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chan, T.F., Wan, W.L.: Analysis of projection methods for solving linear systems with multiple right-hand sides. SIAM J. Sci. Comput. 18(6), 1698–1721 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen, J.: A deflated version of the block conjugate gradient algorithm with an application to Gaussian process maximum likelihood estimation. Preprint, ANL/MCS-P1927-0811, Argonne National Laboratory, Argonne, IL (2011)

  8. Cockett, R.: The block conjugate gradient for multiple right hand sides in a direct current resistivity inversion. http://www.row1.ca/s/pdfs/courses/BlockCG.pdf. Accessed 28 Feb 2015

  9. Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 1–25. https://www.cise.ufl.edu/research/sparse/matrices/ (2011). Accessed 28 Feb 2015

  10. Dubrulle, A.A.: Retooling the method of block conjugate gradients. Electron. Trans. Numer. Anal. 12, 216–233 (2001)

    MathSciNet  MATH  Google Scholar 

  11. Duff, I.S., Grimes, R.G., Lewis, J.G.: Users’ guide for the Harwell–Boeing sparse matrix collection (Release I). Tech. rep., TR/PA/92/86, CERFACS, Toulouse, France. http://math.nist.gov/MatrixMarket/ (1992). Accessed 28 Feb 2015

  12. El Guennouni, A., Jbilou, K., Sadok, H.: A block version of BiCGSTAB for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 16, 129–142 (2003)

    MathSciNet  MATH  Google Scholar 

  13. Erhel, J., Guyomarc’h, F.: An augmented conjugate gradient method for solving consecutive symmetric positive definite linear systems. SIAM J. Matrix Anal. Appl. 21(4), 1279–1299 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  14. Feng, Y.T., Owen, D.R.J., Perić, D.: A block conjugate gradient method applied to linear systems with multiple right-hand sides. Comput. Methods Appl. Mech. Eng. 127(1), 203–215 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  15. Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Numerical Analysis of Proceedings of 6th Biennial Dundee Conference, University of Dundee, Dundee, 1975, Lecture notes in mathematics, vol. 506, pp. 73–89. Springer, Berlin (1976)

  16. Freund, R.W., Malhotra, M.: A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides. Linear Algebra Appl. 254(1), 119–157 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  17. Freund, R.W., Nachtigal, N.M.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numer. Math. 60(1), 315–339 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  18. Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34(2), 495–518 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  19. Golub, G.H., van Loan, C.F.: Matrix computations, 4th edn. Johns Hopkins University Press, Baltimore (2012)

    MATH  Google Scholar 

  20. Gutknecht, M.H.: Block Krylov space solvers: a survey. Tech. rep. http://www.sam.math.ethz.ch/~mhg/talks/bkss.pdf (2005). Accessed 28 Feb 2015

  21. Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern mathematical models, methods and algorithms for real world systems, pp. 420–447. Anamaya Publishers, New Delhi (2007)

    Google Scholar 

  22. Gutknecht, M.H., Schmelzer, T.: The block grade of a block Krylov space. Linear Algebra Appl. 430(1), 174–185 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  23. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)

    Article  MathSciNet  MATH  Google Scholar 

  24. Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31(1), 49–63 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  25. Jbilou, K., Sadok, H., Tinzefte, A.: Oblique projection methods for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 20, 119–138 (2005)

    MathSciNet  MATH  Google Scholar 

  26. Ji, H., Sosonkina, M., Li, Y.: An implementation of block conjugate gradient algorithm on CPU–GPU processors. In: Proceedings of the 1st International Workshop on Hardware–Software Co–Design for High Performance Computing, IEEE Press, pp. 72–77 (2014)

  27. Kaasschieter, E.F.: Preconditioned conjugate gradients for solving singular systems. J. Comput. Appl. Math. 24(1), 265–275 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  28. Langou, J.: Iterative methods for solving linear systems with multiple right-hand sides. Ph.D. thesis, CERFACS, France (2003)

  29. Leyk, Z.: Breakdowns and stagnation in iterative methods. BIT Numer. Math. 37(2), 377–403 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  30. Li, G.: A block variant of the GMRES method on massively parallel processors. Parallel Comput. 23(8), 1005–1019 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  31. Li, N., Saad, Y., Chow, E.: Crout versions of ILU for general sparse matrices. SIAM J. Sci. Comput. 25(2), 716–728 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  32. Li, Y.: Matrices for the near-breakdown example. http://www.cs.odu.edu/~yaohang/publications/BFBCGappendix.pdf (2016). Accessed 30 Aug 2015

  33. Lim, J.S., Un, C.K.: Block conjugate gradient algorithms for adaptive filtering. Signal Process. 55(1), 65–77 (1996)

    Article  MATH  Google Scholar 

  34. McCarthy, J.F.: Block-conjugate-gradient method. Phys. Rev. D. 40(6), 2149–2152 (1989)

    Article  Google Scholar 

  35. Nakamura, Y., Ishikawa, K.I., Kuramashi, Y., Sakurai, T., Tadano, H.: Modified block BiCGSTAB for lattice QCD. Comput. Phys. Comm. 183(1), 34–37 (2012)

    Article  Google Scholar 

  36. Nicolaides, R.A.: Deflation of conjugate gradients with applications to boundary value problems. SIAM J. Numer. Anal. 24(2), 355–365 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  37. Nikishin, A.A., Yeremin, A.Y.: Variable block CG algorithms for solving large sparse symmetric positive definite linear systems on parallel computers, I: General iterative scheme. SIAM J. Matrix Anal. Appl. 16(4), 1135–1153 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  38. O’Leary, D.P.: The block conjugate gradient algorithm and related methods. Linear Algebra Appl. 29, 293–322 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  39. O’Leary, D.P.: Parallel implementation of the block conjugate gradient algorithm. Parallel Comput. 5(1), 127–139 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  40. O’Leary, D.P., Widlund, O.: Capacitance matrix methods for the Helmholtz equation on general three-dimensional regions. Math. Comp. 33, 849–879 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  41. Parlett, B.N.: A new look at the Lanczos algorithm for solving symmetric systems of linear equations. Linear Algebra Appl. 29, 323–346 (1980)

    Article  MathSciNet  MATH  Google Scholar 

  42. Reid, J.K.: On the method of conjugate gradients for the solution of large sparse systems of linear equations. In: Reid, J.K. (ed.) Large sparse sets of linear equations, pp. 231–254. Academic Press, New York (1971)

    Google Scholar 

  43. Robbé, M., Sadkane, M.: Exact and inexact breakdowns in the block GMRES method. Linear Algebra Appl. 419(1), 265–285 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  44. Rutishauser, H.: Theory of gradient methods. In: Refined iterative methods for computation of the solution and the eigenvalues of self–adjoint boundary value problems. Basel–Stuttgart, Institute of Applied Mathematics, Birkhäuser Verlag, Zurich, pp. 24–49 (1959)

  45. Saad, Y.: On the Lanczos method for solving symmetric linear systems with several right-hand sides. Math. Comput. 48(178), 651–662 (1987)

    MATH  Google Scholar 

  46. Saad, Y.: Iterative methods for sparse linear systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  47. Saad, Y., Yeung, M., Erhel, J., Guyomarc’h, F.: A deflated version of the conjugate gradient algorithm. SIAM J. Sci. Comput. 21(5), 1909–1926 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  48. Salkuyeh, D.K.: CG-type algorithms to solve symmetric matrix equations. Appl. Math. Comput. 172(2), 985–999 (2006)

    MathSciNet  MATH  Google Scholar 

  49. Schmelzer, T.: Block Krylov methods for Hermitian linear systems. Diploma thesis, Department of Mathematics, University of Kaiserslautern, Germany (2004)

  50. Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (1994)

  51. Simoncini, V., Gallopoulos, E.: An iterative method for nonsymmetric systems with multiple right-hand sides. SIAM J. Sci. Comput. 16(4), 917–933 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  52. Simoncini, V., Gallopoulos, E.: Convergence properties of block GMRES and matrix polynomials. Linear Algebra Appl. 247, 97–119 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  53. Smith, C.F., Peterson, A.F., Mittra, R.: A conjugate gradient algorithm for the treatment of multiple incident electromagnetic fields. IEEE Trans. Antennas Propag. 37(11), 1490–1493 (1989)

    Article  Google Scholar 

  54. Towns, J., Cockerill, T., Dahan, M., Foster, I., Gaither, K., Grimshaw, A., Hazlewood, V., Lathrop, S., Lifka, D., Peterson, G.D., Roskies, R., Scott, J.R., Wilkins-Diehr, N.: XSEDE: accelerating scientific discovery. Comput. Sci. Eng. 16(5), 62–74 (2014)

    Article  Google Scholar 

  55. Vital, B.: Etude de quelques méthodes de résolution de problèmes linéaires de grande taille sur multiprocesseur. Ph.D. thesis, Université de Rennes I, Rennes, France (1990)

  56. Van der Vorst, H.A.: An iterative solution method for solving \(f(A)x=b\), using Krylov subspace information obtained for the symmetric positive definite matrix \(A\). J. Comput. Appl. Math. 18(2), 249–263 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  57. Zavorin, I., O’Leary, D.P., Elman, H.: Complete stagnation of GMRES. Linear Algebra Appl. 367, 165–183 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  58. Zhang, J., Dai, H., Zhao, J.: A new family of global methods for linear systems with multiple right-hand sides. J. Comput. Appl. Math. 236(6), 1562–1575 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

Y. Li acknowledges support from National Science Foundation through Grant No. CCF-1066471. H. Ji acknowledges support from Old Dominion University Modeling and Simulation Fellowship. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation Grant No. ACI-1053575. The authors would like to thank Dr. Michiel E. Hochstenbach, Dr. Martin H. Gutkne-cht, and other reviewers for their very helpful comments and suggestions on this work.

Author information

Authors and Affiliations

  1. Department of Computer Science, California State Polytechnic University, Pomona, CA, 91768, USA

    Hao Ji

  2. Department of Computer Science, Old Dominion University, Norfolk, VA, 23529, USA

    Yaohang Li

Authors
  1. Hao Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yaohang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yaohang Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, H., Li, Y. A breakdown-free block conjugate gradient method. Bit Numer Math 57, 379–403 (2017). https://doi.org/10.1007/s10543-016-0631-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-016-0631-z

Keywords

Mathematics Subject Classification

Search

Navigation