Advances in Computational Mathematics

, Volume 6, Issue 1, pp 1–24 | Cite as

Iterative methods for solvingAx=b, GMRES/FOM versus QMR/BiCG

  • Jane Cullum


We study the convergence of the GMRES/FOM and QMR/BiCG methods for solving nonsymmetric systems of equationsAx=b. We prove, in exact arithmetic, that any type of residual norm convergence obtained using BiCG can also be obtained using FOM but on a different system of equations. We consider practical comparisons of these procedures when they are applied to the same matrices. We use a unitary invariance shared by both methods, to construct test matrices where we can vary the nonnormality of the test matrix by variations in simplified eigenvector matrices. We used these test problems in two sets of numerical experiments. The first set of experiments was designed to study effects of increasing nonnormality on the convergence of GMRES and QMR. The second set of experiments was designed to track effects of the eigenvalue distribution on the convergence of QMR. In these tests the GMRES residual norms decreased significantly more rapidly than the QMR residual norms but without corresponding decreases in the error norms. Furthermore, as the nonnormality ofA was increased, the GMRES residual norms decreased more rapidly. This led to premature termination of the GMRES procedure on highly nonnormal problems. On the nonnormal test problems the QMR residual norms exhibited less sensitivity to changes in the nonnormality. The convergence of either type of procedure, as measured by the error norms, was delayed by the presence of large or small outliers and affected by the type of eigenvalues, real or complex, in the eigenvalue distribution ofA. For GMRES this effect can be seen only in the error norm plots.


Numerical Experiment Iterative Method Test Problem Error Norm Premature Termination 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    R. Barrett et al.,Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods (SIAM, Philadelphia, PA, 1994).Google Scholar
  2. [2]
    T. Braconnier, Influence of orthogonality on the backward error and the stopping criterion for Krylov methods, University of Manchester, Numerical Analysis Report No. 281, Manchester Centre for Computational Mathematics, University of Manchester, Manchester, England (December 1995).Google Scholar
  3. [3]
    T. Braconnier, F. Chatelin and V. Fraysse, The influence of large nonnormality on the quality of convergence of iterative methods in linear algebra, CERFACS Technical Report TR-PA-94-07, CERFACS, Toulouse, France (1994).Google Scholar
  4. [4]
    C. Brezinski, M. Redivo Zaglia and H. Sadok, A breakdown-free Lanczos type algorithm for solving linear systems, Numer. Math. 63 (1992) 29–38.CrossRefGoogle Scholar
  5. [5]
    P. N. Brown, A theoretical comparison of the Arnoldi and GMRES algorithms, SIAM J. Sci. Statist. Comput. 20 (1991) 58–78.CrossRefGoogle Scholar
  6. [6]
    F. Chaitin-Chatelin and V. Frayseé,Lectures on Finite Precision Computation (SIAM, Philadelphia, PA, 1996).Google Scholar
  7. [7]
    F. Chaitin-Chatelin, Finite precision computations, reliability of numerical software, CERFACS Technical Report TR-PA-94-05, CERFACS, Toulouse, France (1994).Google Scholar
  8. [8]
    F. Chatelin and V. Fraysse, numerical illustrations by T. Braconnier, Qualitative computing, elements for a theory for finite precision computation, CERFACS Technical Report TR-PA-93-12, CERFACS, Toulouse, France (1993).Google Scholar
  9. [9]
    J. Cullum and A. Greenbaum, Relations between Galerkin and norm-minimizing iterative methods for solving linear systems, SIAM J. Matrix Anal. Appl. 17(2) (1996) 223–247.CrossRefGoogle Scholar
  10. [10]
    J. Cullum, Iterative methods for solvingAx=b, GMRES versus QMR/BICG, IBM Research Report, RC20325, IBM Research, Yorktown Heights, NY (January 1996).Google Scholar
  11. [11]
    J. Cullum, Arnoldi versus nonsymmetric Lanczos algorithms for solving matrix eigenvalue problems, BIT 36 (1996) 470–493.CrossRefGoogle Scholar
  12. [12]
    J. Cullum and R. Willoughby, A practical procedure for computing eigenvalues of large sparse nonsymmetric matrices, in:Large Scale Eigenvalue Problems, eds. J. Cullum and R. Willoughby (North-Holland, Amsterdam, 1986) pp. 193–240.Google Scholar
  13. [13]
    R. W. Freund, M. H. Gutknecht and N. M. Nachtigal, An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices, SIAM J. Sci. Statist. Comput. 14 (1993) 137–158.CrossRefGoogle Scholar
  14. [14]
    R. Freund and N. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM J. Sci. Comput. 15 (1994) 313–337.CrossRefGoogle Scholar
  15. [15]
    R. W. Freund and N. M. Nachtigal, QMR: a quasi-minimal residual method for non-Hermitian linear systems, Numer. Math. 60 (1991) 315–339.CrossRefGoogle Scholar
  16. [16]
    A. Greenbaum, private communication.Google Scholar
  17. [17]
    A. Greenbaum, V. Ptak and Z. Strakos, Any nonincreasing convergence curve is possible for GMRES, SIAM J. Matrix Anal. Appl. (1996).Google Scholar
  18. [18]
    A. Greenbaum and Z. Strakos, Matrices that generate the same Krylov residual spaces, in:Recent Advances in Iterative Methods, eds. G. Golub, A. Greenbaum and M. Luskin, IMA Volumes in Mathematics and its Applications 60 (Springer, Berlin, 1993) pp. 95–118.Google Scholar
  19. [19]
    G. H. Golub and C. F. Van Loan,Matrix Computations (Johns Hopkins University Press, Baltimore, MD, 2nd ed., 1989).Google Scholar
  20. [20]
    M. H. Gutknecht, Variants of BiCGSTAB for matrices with complex spectrum, SIAM J. Sci. Statist. Comput. 14 (1993) 1020–1033.CrossRefGoogle Scholar
  21. [21]
    C. Moler et al.,MATLAB User's Guide (MathWorks, Natick, MA, 1992).Google Scholar
  22. [22]
    N. M. Nachtigal, L. Reichel and L. N. Trefethen, A hybrid GMRES algorithm for nonsymmetric linear systems, SIAM J. Matrix Anal. Appl. 13 (1992) 796–825.CrossRefGoogle Scholar
  23. [23]
    B. N. Parlett, D. R. Taylor and Z. A. Liu, A look-ahead Lanczos algorithm for unsymmetric matrices, Math. Comp. 44 (1985) 105–124.Google Scholar
  24. [24]
    Y. Saad,Iterative Methods for Sparse Linear Systems (PWS Publishing Co., Boston, MA, 1996).Google Scholar
  25. [25]
    Y. Saad and M. H. Schultz, GMRES: A generalized minimum residual algorithm for solving non-symmetric linear systems, SIAM J. Sci. Statist. Comput. 7 (1986) 856–869.CrossRefGoogle Scholar
  26. [26]
    Y. Saad, Krylov subspace methods for solving unsymmetric linear systems, Math. Comp. 37 (1981) 105–126.Google Scholar
  27. [27]
    L. N. Trefethen, A. E. Trefethen, S. C. Reddy and T. A. Driscoll, Hydrodynamic stability without eigenvalues, Science 261 (1993) 578–584.MathSciNetGoogle Scholar
  28. [28]
    L. N. Trefethen, Pseudospectra of matrices, in:Numerical Analysis 1991, eds. D. F. Griffiths and G. A. Watson (Longman Scientific and Technical, Harlow, UK, 1992).Google Scholar
  29. [29]
    L. N. Trefethen, Approximation theory and numerical linear algebra, in:Algorithms for Approximation II, eds. J. C. Mason and M. G. Cox (Chapman and Hall, London, 1990) pp. 336–360.Google Scholar
  30. [30]
    G. L. G. Sleijpen, H. A. van der Vorst and D. R. Fokkema, BiCGSTAB(1) and other hybrid Bi-CG methods, Numerical Algorithms 7 (1994) 75–109.Google Scholar
  31. [31]
    H. A. van der Vorst, A fast and smoothly converging variant of BiCG for the solutions of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 13 (1992) 631–644.CrossRefGoogle Scholar
  32. [32]
    H. F. Walker, Implementations of the GMRES method using Householder transformations, SIAM J. Sci. Statist. Comput. 9(1) (1988) 152–163.CrossRefGoogle Scholar

Copyright information

© J.C. Baltzer AG, Science Publishers 1996

Authors and Affiliations

  • Jane Cullum
    • 1
  1. 1.Mathematical Sciences Department, IBM Research DivisionT.J. Watson Research CenterYorktown HeightsUSA

Personalised recommendations