Abstract
We study the convergence of the GMRES/FOM and QMR/BiCG methods for solving nonsymmetric systems of equationsAx=b. We prove, in exact arithmetic, that any type of residual norm convergence obtained using BiCG can also be obtained using FOM but on a different system of equations. We consider practical comparisons of these procedures when they are applied to the same matrices. We use a unitary invariance shared by both methods, to construct test matrices where we can vary the nonnormality of the test matrix by variations in simplified eigenvector matrices. We used these test problems in two sets of numerical experiments. The first set of experiments was designed to study effects of increasing nonnormality on the convergence of GMRES and QMR. The second set of experiments was designed to track effects of the eigenvalue distribution on the convergence of QMR. In these tests the GMRES residual norms decreased significantly more rapidly than the QMR residual norms but without corresponding decreases in the error norms. Furthermore, as the nonnormality ofA was increased, the GMRES residual norms decreased more rapidly. This led to premature termination of the GMRES procedure on highly nonnormal problems. On the nonnormal test problems the QMR residual norms exhibited less sensitivity to changes in the nonnormality. The convergence of either type of procedure, as measured by the error norms, was delayed by the presence of large or small outliers and affected by the type of eigenvalues, real or complex, in the eigenvalue distribution ofA. For GMRES this effect can be seen only in the error norm plots.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R. Barrett et al.,Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods (SIAM, Philadelphia, PA, 1994).
T. Braconnier, Influence of orthogonality on the backward error and the stopping criterion for Krylov methods, University of Manchester, Numerical Analysis Report No. 281, Manchester Centre for Computational Mathematics, University of Manchester, Manchester, England (December 1995).
T. Braconnier, F. Chatelin and V. Fraysse, The influence of large nonnormality on the quality of convergence of iterative methods in linear algebra, CERFACS Technical Report TR-PA-94-07, CERFACS, Toulouse, France (1994).
C. Brezinski, M. Redivo Zaglia and H. Sadok, A breakdown-free Lanczos type algorithm for solving linear systems, Numer. Math. 63 (1992) 29–38.
P. N. Brown, A theoretical comparison of the Arnoldi and GMRES algorithms, SIAM J. Sci. Statist. Comput. 20 (1991) 58–78.
F. Chaitin-Chatelin and V. Frayseé,Lectures on Finite Precision Computation (SIAM, Philadelphia, PA, 1996).
F. Chaitin-Chatelin, Finite precision computations, reliability of numerical software, CERFACS Technical Report TR-PA-94-05, CERFACS, Toulouse, France (1994).
F. Chatelin and V. Fraysse, numerical illustrations by T. Braconnier, Qualitative computing, elements for a theory for finite precision computation, CERFACS Technical Report TR-PA-93-12, CERFACS, Toulouse, France (1993).
J. Cullum and A. Greenbaum, Relations between Galerkin and norm-minimizing iterative methods for solving linear systems, SIAM J. Matrix Anal. Appl. 17(2) (1996) 223–247.
J. Cullum, Iterative methods for solvingAx=b, GMRES versus QMR/BICG, IBM Research Report, RC20325, IBM Research, Yorktown Heights, NY (January 1996).
J. Cullum, Arnoldi versus nonsymmetric Lanczos algorithms for solving matrix eigenvalue problems, BIT 36 (1996) 470–493.
J. Cullum and R. Willoughby, A practical procedure for computing eigenvalues of large sparse nonsymmetric matrices, in:Large Scale Eigenvalue Problems, eds. J. Cullum and R. Willoughby (North-Holland, Amsterdam, 1986) pp. 193–240.
R. W. Freund, M. H. Gutknecht and N. M. Nachtigal, An implementation of the look-ahead Lanczos algorithm for non-Hermitian matrices, SIAM J. Sci. Statist. Comput. 14 (1993) 137–158.
R. Freund and N. Nachtigal, An implementation of the QMR method based on coupled two-term recurrences, SIAM J. Sci. Comput. 15 (1994) 313–337.
R. W. Freund and N. M. Nachtigal, QMR: a quasi-minimal residual method for non-Hermitian linear systems, Numer. Math. 60 (1991) 315–339.
A. Greenbaum, private communication.
A. Greenbaum, V. Ptak and Z. Strakos, Any nonincreasing convergence curve is possible for GMRES, SIAM J. Matrix Anal. Appl. (1996).
A. Greenbaum and Z. Strakos, Matrices that generate the same Krylov residual spaces, in:Recent Advances in Iterative Methods, eds. G. Golub, A. Greenbaum and M. Luskin, IMA Volumes in Mathematics and its Applications 60 (Springer, Berlin, 1993) pp. 95–118.
G. H. Golub and C. F. Van Loan,Matrix Computations (Johns Hopkins University Press, Baltimore, MD, 2nd ed., 1989).
M. H. Gutknecht, Variants of BiCGSTAB for matrices with complex spectrum, SIAM J. Sci. Statist. Comput. 14 (1993) 1020–1033.
C. Moler et al.,MATLAB User's Guide (MathWorks, Natick, MA, 1992).
N. M. Nachtigal, L. Reichel and L. N. Trefethen, A hybrid GMRES algorithm for nonsymmetric linear systems, SIAM J. Matrix Anal. Appl. 13 (1992) 796–825.
B. N. Parlett, D. R. Taylor and Z. A. Liu, A look-ahead Lanczos algorithm for unsymmetric matrices, Math. Comp. 44 (1985) 105–124.
Y. Saad,Iterative Methods for Sparse Linear Systems (PWS Publishing Co., Boston, MA, 1996).
Y. Saad and M. H. Schultz, GMRES: A generalized minimum residual algorithm for solving non-symmetric linear systems, SIAM J. Sci. Statist. Comput. 7 (1986) 856–869.
Y. Saad, Krylov subspace methods for solving unsymmetric linear systems, Math. Comp. 37 (1981) 105–126.
L. N. Trefethen, A. E. Trefethen, S. C. Reddy and T. A. Driscoll, Hydrodynamic stability without eigenvalues, Science 261 (1993) 578–584.
L. N. Trefethen, Pseudospectra of matrices, in:Numerical Analysis 1991, eds. D. F. Griffiths and G. A. Watson (Longman Scientific and Technical, Harlow, UK, 1992).
L. N. Trefethen, Approximation theory and numerical linear algebra, in:Algorithms for Approximation II, eds. J. C. Mason and M. G. Cox (Chapman and Hall, London, 1990) pp. 336–360.
G. L. G. Sleijpen, H. A. van der Vorst and D. R. Fokkema, BiCGSTAB(1) and other hybrid Bi-CG methods, Numerical Algorithms 7 (1994) 75–109.
H. A. van der Vorst, A fast and smoothly converging variant of BiCG for the solutions of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput. 13 (1992) 631–644.
H. F. Walker, Implementations of the GMRES method using Householder transformations, SIAM J. Sci. Statist. Comput. 9(1) (1988) 152–163.
Author information
Authors and Affiliations
Additional information
Communicated by C.A. Micchelli
In honor of the 70th birthday of Ted Rivlin
This work was supported by NSF grant GER-9450081.
Rights and permissions
About this article
Cite this article
Cullum, J. Iterative methods for solvingAx=b, GMRES/FOM versus QMR/BiCG. Adv Comput Math 6, 1–24 (1996). https://doi.org/10.1007/BF02127693
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02127693