Abstract
In this paper, we analyze all possible situations of rank deficiency that cause breakdown in block conjugate gradient (BCG) solvers. A simple solution, breakdown-free block conjugate gradient (BFBCG), is designed to address the rank deficiency problem. The rationale of the BFBCG algorithm is to derive new forms of parameter matrices based on the potentially reduced search subspace to handle rank deficiency. Orthogonality properties and convergence of BFBCG in case of rank deficiency are justified accordingly with mathematical rigor. BFBCG yields faster convergence than restarting BCG when breakdown occurs. Numerical examples suffering from rank deficiency are provided to demonstrate the robustness of BFBCG.
Similar content being viewed by others
References
Agullo, E., Giraud, L., Jing, Y.F.: Block GMRES method with inexact breakdowns and deflated restarting. SIAM J. Matrix Anal. Appl. 35(4), 1625–1651 (2014)
Axelsson, O.: On preconditioning and convergence acceleration in sparse matrix problems. CERN 74–10, Genèva, Switzerland (1974)
Barrett, R., Berry, M., Chan, T., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., van der Vorst, H.: Templates for the solution of linear systems: building blocks for iterative methods. SIAM, Philadelphia (1994)
Birk, S., Frommer, A.: A CG method for multiple right hand sides and multiple shifts in lattice QCD calculations. In: Proceedings of the XXIX International Symposium on Lattice Field Theory, pp. 10–16 (2011)
Broyden, C.G.: A breakdown of the block CG method. Optim. Method. Softw. 7(1), 41–55 (1996)
Chan, T.F., Wan, W.L.: Analysis of projection methods for solving linear systems with multiple right-hand sides. SIAM J. Sci. Comput. 18(6), 1698–1721 (1997)
Chen, J.: A deflated version of the block conjugate gradient algorithm with an application to Gaussian process maximum likelihood estimation. Preprint, ANL/MCS-P1927-0811, Argonne National Laboratory, Argonne, IL (2011)
Cockett, R.: The block conjugate gradient for multiple right hand sides in a direct current resistivity inversion. http://www.row1.ca/s/pdfs/courses/BlockCG.pdf. Accessed 28 Feb 2015
Davis, T.A., Hu, Y.: The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 38(1), 1–25. https://www.cise.ufl.edu/research/sparse/matrices/ (2011). Accessed 28 Feb 2015
Dubrulle, A.A.: Retooling the method of block conjugate gradients. Electron. Trans. Numer. Anal. 12, 216–233 (2001)
Duff, I.S., Grimes, R.G., Lewis, J.G.: Users’ guide for the Harwell–Boeing sparse matrix collection (Release I). Tech. rep., TR/PA/92/86, CERFACS, Toulouse, France. http://math.nist.gov/MatrixMarket/ (1992). Accessed 28 Feb 2015
El Guennouni, A., Jbilou, K., Sadok, H.: A block version of BiCGSTAB for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 16, 129–142 (2003)
Erhel, J., Guyomarc’h, F.: An augmented conjugate gradient method for solving consecutive symmetric positive definite linear systems. SIAM J. Matrix Anal. Appl. 21(4), 1279–1299 (2000)
Feng, Y.T., Owen, D.R.J., Perić, D.: A block conjugate gradient method applied to linear systems with multiple right-hand sides. Comput. Methods Appl. Mech. Eng. 127(1), 203–215 (1995)
Fletcher, R.: Conjugate gradient methods for indefinite systems. In: Numerical Analysis of Proceedings of 6th Biennial Dundee Conference, University of Dundee, Dundee, 1975, Lecture notes in mathematics, vol. 506, pp. 73–89. Springer, Berlin (1976)
Freund, R.W., Malhotra, M.: A block QMR algorithm for non-Hermitian linear systems with multiple right-hand sides. Linear Algebra Appl. 254(1), 119–157 (1997)
Freund, R.W., Nachtigal, N.M.: QMR: a quasi-minimal residual method for non-Hermitian linear systems. Numer. Math. 60(1), 315–339 (1991)
Gaul, A., Gutknecht, M.H., Liesen, J., Nabben, R.: A framework for deflated and augmented Krylov subspace methods. SIAM J. Matrix Anal. Appl. 34(2), 495–518 (2013)
Golub, G.H., van Loan, C.F.: Matrix computations, 4th edn. Johns Hopkins University Press, Baltimore (2012)
Gutknecht, M.H.: Block Krylov space solvers: a survey. Tech. rep. http://www.sam.math.ethz.ch/~mhg/talks/bkss.pdf (2005). Accessed 28 Feb 2015
Gutknecht, M.H.: Block Krylov space methods for linear systems with multiple right-hand sides: an introduction. In: Siddiqi, A.H., Duff, I.S., Christensen, O. (eds.) Modern mathematical models, methods and algorithms for real world systems, pp. 420–447. Anamaya Publishers, New Delhi (2007)
Gutknecht, M.H., Schmelzer, T.: The block grade of a block Krylov space. Linear Algebra Appl. 430(1), 174–185 (2009)
Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49, 409–436 (1952)
Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31(1), 49–63 (1999)
Jbilou, K., Sadok, H., Tinzefte, A.: Oblique projection methods for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 20, 119–138 (2005)
Ji, H., Sosonkina, M., Li, Y.: An implementation of block conjugate gradient algorithm on CPU–GPU processors. In: Proceedings of the 1st International Workshop on Hardware–Software Co–Design for High Performance Computing, IEEE Press, pp. 72–77 (2014)
Kaasschieter, E.F.: Preconditioned conjugate gradients for solving singular systems. J. Comput. Appl. Math. 24(1), 265–275 (1988)
Langou, J.: Iterative methods for solving linear systems with multiple right-hand sides. Ph.D. thesis, CERFACS, France (2003)
Leyk, Z.: Breakdowns and stagnation in iterative methods. BIT Numer. Math. 37(2), 377–403 (1997)
Li, G.: A block variant of the GMRES method on massively parallel processors. Parallel Comput. 23(8), 1005–1019 (1997)
Li, N., Saad, Y., Chow, E.: Crout versions of ILU for general sparse matrices. SIAM J. Sci. Comput. 25(2), 716–728 (2003)
Li, Y.: Matrices for the near-breakdown example. http://www.cs.odu.edu/~yaohang/publications/BFBCGappendix.pdf (2016). Accessed 30 Aug 2015
Lim, J.S., Un, C.K.: Block conjugate gradient algorithms for adaptive filtering. Signal Process. 55(1), 65–77 (1996)
McCarthy, J.F.: Block-conjugate-gradient method. Phys. Rev. D. 40(6), 2149–2152 (1989)
Nakamura, Y., Ishikawa, K.I., Kuramashi, Y., Sakurai, T., Tadano, H.: Modified block BiCGSTAB for lattice QCD. Comput. Phys. Comm. 183(1), 34–37 (2012)
Nicolaides, R.A.: Deflation of conjugate gradients with applications to boundary value problems. SIAM J. Numer. Anal. 24(2), 355–365 (1987)
Nikishin, A.A., Yeremin, A.Y.: Variable block CG algorithms for solving large sparse symmetric positive definite linear systems on parallel computers, I: General iterative scheme. SIAM J. Matrix Anal. Appl. 16(4), 1135–1153 (1995)
O’Leary, D.P.: The block conjugate gradient algorithm and related methods. Linear Algebra Appl. 29, 293–322 (1980)
O’Leary, D.P.: Parallel implementation of the block conjugate gradient algorithm. Parallel Comput. 5(1), 127–139 (1987)
O’Leary, D.P., Widlund, O.: Capacitance matrix methods for the Helmholtz equation on general three-dimensional regions. Math. Comp. 33, 849–879 (1979)
Parlett, B.N.: A new look at the Lanczos algorithm for solving symmetric systems of linear equations. Linear Algebra Appl. 29, 323–346 (1980)
Reid, J.K.: On the method of conjugate gradients for the solution of large sparse systems of linear equations. In: Reid, J.K. (ed.) Large sparse sets of linear equations, pp. 231–254. Academic Press, New York (1971)
Robbé, M., Sadkane, M.: Exact and inexact breakdowns in the block GMRES method. Linear Algebra Appl. 419(1), 265–285 (2006)
Rutishauser, H.: Theory of gradient methods. In: Refined iterative methods for computation of the solution and the eigenvalues of self–adjoint boundary value problems. Basel–Stuttgart, Institute of Applied Mathematics, Birkhäuser Verlag, Zurich, pp. 24–49 (1959)
Saad, Y.: On the Lanczos method for solving symmetric linear systems with several right-hand sides. Math. Comput. 48(178), 651–662 (1987)
Saad, Y.: Iterative methods for sparse linear systems, 2nd edn. SIAM, Philadelphia (2003)
Saad, Y., Yeung, M., Erhel, J., Guyomarc’h, F.: A deflated version of the conjugate gradient algorithm. SIAM J. Sci. Comput. 21(5), 1909–1926 (2000)
Salkuyeh, D.K.: CG-type algorithms to solve symmetric matrix equations. Appl. Math. Comput. 172(2), 985–999 (2006)
Schmelzer, T.: Block Krylov methods for Hermitian linear systems. Diploma thesis, Department of Mathematics, University of Kaiserslautern, Germany (2004)
Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., School of Computer Science, Carnegie Mellon University, Pittsburgh, PA (1994)
Simoncini, V., Gallopoulos, E.: An iterative method for nonsymmetric systems with multiple right-hand sides. SIAM J. Sci. Comput. 16(4), 917–933 (1995)
Simoncini, V., Gallopoulos, E.: Convergence properties of block GMRES and matrix polynomials. Linear Algebra Appl. 247, 97–119 (1996)
Smith, C.F., Peterson, A.F., Mittra, R.: A conjugate gradient algorithm for the treatment of multiple incident electromagnetic fields. IEEE Trans. Antennas Propag. 37(11), 1490–1493 (1989)
Towns, J., Cockerill, T., Dahan, M., Foster, I., Gaither, K., Grimshaw, A., Hazlewood, V., Lathrop, S., Lifka, D., Peterson, G.D., Roskies, R., Scott, J.R., Wilkins-Diehr, N.: XSEDE: accelerating scientific discovery. Comput. Sci. Eng. 16(5), 62–74 (2014)
Vital, B.: Etude de quelques méthodes de résolution de problèmes linéaires de grande taille sur multiprocesseur. Ph.D. thesis, Université de Rennes I, Rennes, France (1990)
Van der Vorst, H.A.: An iterative solution method for solving \(f(A)x=b\), using Krylov subspace information obtained for the symmetric positive definite matrix \(A\). J. Comput. Appl. Math. 18(2), 249–263 (1987)
Zavorin, I., O’Leary, D.P., Elman, H.: Complete stagnation of GMRES. Linear Algebra Appl. 367, 165–183 (2003)
Zhang, J., Dai, H., Zhao, J.: A new family of global methods for linear systems with multiple right-hand sides. J. Comput. Appl. Math. 236(6), 1562–1575 (2011)
Acknowledgments
Y. Li acknowledges support from National Science Foundation through Grant No. CCF-1066471. H. Ji acknowledges support from Old Dominion University Modeling and Simulation Fellowship. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation Grant No. ACI-1053575. The authors would like to thank Dr. Michiel E. Hochstenbach, Dr. Martin H. Gutkne-cht, and other reviewers for their very helpful comments and suggestions on this work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ji, H., Li, Y. A breakdown-free block conjugate gradient method. Bit Numer Math 57, 379–403 (2017). https://doi.org/10.1007/s10543-016-0631-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10543-016-0631-z
Keywords
- Rank deficiency
- Breakdown-free block conjugate gradient method
- Block Krylov subspace
- Multiple right-hand sides
- Near-breakdown problem