BIT Numerical Mathematics

, Volume 37, Issue 2, pp 377–403

Breakdowns and stagnation in iterative methods

  • Zbigniew Leyk


One of the disadvantages of Krylov subspace iterative methods is the possibility of breakdown. This occurs when it is impossible to get the next approximation of the solution to the linear system of equationsAu=f. There are two different situations: lucky breakdown, when we have found the solution and hard breakdown, when the next Krylov subspace cannot be generated and/or the next approximate solution (iterate) cannot be computed. We show that some breakdowns depend on the chosen method of generating the basis vectors. Another undesirable feature of the iterative methods is stagnation. This occurs when the error does not change for several iterative steps. We investigate when iterative methods can stagnate and describe conditions which characterize stagnation. We show that in some cases stagnation can imply breakdown.

AMS subject classification


Key words

Iterative methods Krylov subspace biconjugate gradient conjugate gradient GMRES generalized conjugate residual 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S. Ashby, T. Manteuffel, and P. Saylor,A taxonomy for conjugate gradient methods, SIAM J. Numer. Anal., 27 (1990), pp. 1542–1568.MATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    S. Barnett,Matrices. Methods and Applications, Clarendon Press, Oxford, 1990.MATHGoogle Scholar
  3. 3.
    R. Barrett, M. Berry, T. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H. van der Vorst,Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM, Philadelphia, 1993.MATHGoogle Scholar
  4. 4.
    C. Brezinski, M. Redivo Zaglia, and H. Sadok,A breakdown-free Lanczos type algorithm for solving linear systems, Numer. Math., 63 (1992), pp. 29–38.MATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    P. Brown,A theoretical comparison of the Arnoldi and GMRES algorithms, SIAM J. Sci. Stat. Comput., 12 (1991), pp. 58–78.MATHCrossRefGoogle Scholar
  6. 6.
    S. Eisenstat, H. Elman, and M. Schultz,Variational iterative methods for nonsymmetric systems of linear equations, SIAM J. Numer. Anal., 20 (1983), pp. 345–357.MATHMathSciNetCrossRefGoogle Scholar
  7. 7.
    V. Faber and T. Manteuffel,Necessary and sufficient conditions for the existence of a conjugate gradient method, SIAM J. Numer. Anal., 21 (1984), pp. 352–362.MATHMathSciNetCrossRefGoogle Scholar
  8. 8.
    V. Faber and T. Manteuffel,Orthogonal error methods, SIAM J. Numer. Anal., 24 (1987), pp. 170–187.MATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    R., Fletcher,Conjugate gradient methods for indefinite systems, in Proceedings Dundee Conference on Numerical Analysis, 1975, G. Watson, ed., Springer-Verlag, 1976, Lecture Notes in Mathematics 506, pp. 73–89.Google Scholar
  10. 10.
    R. Freund, G. Golub, and N. Nachtigal,Iterative solution of linear systems, Acta Numerica, 1 (1991), pp. 57–100.MathSciNetCrossRefGoogle Scholar
  11. 11.
    R. Freund, M. Gutknecht, and N. Nachtigal,An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices, SIAM J. Sci. Comput., 14 (1993), pp. 137–158.MATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    R. Freund and N. Nachtigal,QMR: a quasi minimal residual method for non-Hermitian linear systems, Numer. Math., 60 (1991), pp. 315–339.MATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    G. Golub and D. O’Leary,Some history of the conjugate gradient and Lanczos methods, SIAM Rev., 31 (1989), pp. 50–102.MATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    G. Golub and C. Van Loan,Matrix Computations 2nd ed., Johns Hopkins Univ. Press, Baltimore, 1989.MATHGoogle Scholar
  15. 15.
    M. Gutknecht,A completed theory of the unsymmetric Lanczos process and related algorithms, part I, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 594–639.MATHMathSciNetCrossRefGoogle Scholar
  16. 16.
    M. Gutknecht,Changing the norm in conjugate gradient type algorithms, SIAM J. Numer. Anal., 30 (1993), pp. 40–56.MATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    M. Gutknecht,A completed theory of the unsymmetric Lanczos process and related algorithms, part II, SIAM J. Matrix Anal. Appl., 15 (1994), pp. 15–58.MATHMathSciNetCrossRefGoogle Scholar
  18. 18.
    M. Hestenes,A History of Scientific Computing, Addison-Wesley, Reading, MA, 1990, Chap. Conjugacy and gradients, pp. 167–179.Google Scholar
  19. 19.
    M. Hestenes and E. Stiefel,Methods of conjugate gradients for solving linear systems, J. Res. Nat. Bur. Standards, 49 (1952), pp. 409–435.MATHMathSciNetGoogle Scholar
  20. 20.
    K. Jea and D. Young,Conjugate-gradient methods for nonsymmetrizable linear systems, Linear Algebra Appl., 52/53 (1983), pp. 399–417.MathSciNetGoogle Scholar
  21. 21.
    W. Joubert,Lanczos methods for the solution of nonsymmetric systems of linear equations, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 926–943.MATHMathSciNetCrossRefGoogle Scholar
  22. 22.
    W. Joubert and T. Manteuffel,Iterative methods for nonsymmetric linear systems, in Iterative Methods for Large Linear Systems, L. Hayes and D. Kincaid, eds., Academic Press, 1990, pp. 149–171.Google Scholar
  23. 23.
    W. Joubert and D. Young,Necessary and sufficient conditions for the simplification of generalized conjugate-gradient algorithms, Linear Algebra Appl., 88/89 (1987), pp. 449–485.MathSciNetCrossRefGoogle Scholar
  24. 24.
    C. Lanczos,An iteration method for the solution of the eigenvalue problem for linear differential and integral operators, J. Res. Nat. Bur. Standards, 45 (1950), pp. 255–282.MathSciNetGoogle Scholar
  25. 25.
    Z. Leyk,Modified generalized conjugate residual method for nonsymmetric systems of linear equations, in Proceedings of the 6th Conference on Computational Techniques and Applications: CTAC93, D. Steward, H. Gardner, and D. Singleton, eds., World Scientific, Singapore, 1994, pp. 338–344.Google Scholar
  26. 26.
    N. Nachtigal, S. Reddy, and L. Trefethen,How fast are nonsymmetric matrix iteration?, SIAM J. Matrix Anal. Appl., 13 (1992), pp. 778–795.MATHMathSciNetCrossRefGoogle Scholar
  27. 27.
    B. Parlett, D. Taylor, and Z. Liu,A look-ahead Lanczos algorithm for unsymmetric matrices, Math. Comp., 44 (1985), pp. 105–124.MATHMathSciNetCrossRefGoogle Scholar
  28. 28.
    Y. Saad,Krylov subspace methods for solving, unsymmetric linear systems, Math. Comp., 37 (1981), pp. 105–126.MATHMathSciNetCrossRefGoogle Scholar
  29. 29.
    Y. Saad,The Lanczos biorthogonalization algorithm and other oblique projection methods for solving nonsymmetric linear systems, SIAM J. Numer. Anal., 19 (1982), pp. 485–506.MATHMathSciNetCrossRefGoogle Scholar
  30. 30.
    Y. Saad and M. Schultz,Conjugate gradient-like algorithms for nonsymmetric systems, Math. Comp., 44 (1985), pp. 417–424.MATHMathSciNetCrossRefGoogle Scholar
  31. 31.
    Y. Saad and M. Schultz,GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856–869.MATHMathSciNetCrossRefGoogle Scholar
  32. 32.
    P. Sonneveld,CGS, a fast Lanczos-type solver for nonsymmetric linear, systems, SIAM J. Sci. Stat. Comput., 10 (1989), pp. 36–52.MATHMathSciNetCrossRefGoogle Scholar
  33. 33.
    H. van der Vorst,A fast and smoothly converging variant of Bi-CG for the solution of nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 13 (1992), pp. 631–644.MATHCrossRefGoogle Scholar
  34. 34.
    H. Walker and L. Zhou,A simpler GMRES, Numer. Linear Algebra Appl., 1 (1994), pp. 571–581.MATHMathSciNetCrossRefGoogle Scholar

Copyright information

© BIT Foundaton 1997

Authors and Affiliations

  • Zbigniew Leyk
    • 1
  1. 1.Institute for Scientific ComputationTexas A&M UniversityCollege StationUSA

Personalised recommendations