Numerical Algorithms

, Volume 68, Issue 1, pp 143–165 | Cite as

The role eigenvalues play in forming GMRES residual norms with non-normal matrices

Original Paper


In this paper we give explicit expressions for the norms of the residual vectors generated by the GMRES algorithm applied to a non-normal matrix. They involve the right-hand side of the linear system, the eigenvalues, the eigenvectors and, in the non-diagonalizable case, the principal vectors. They give a complete description of how eigenvalues contribute in forming residual norms and offer insight in what quantities can prevent GMRES from being governed by eigenvalues.


GMRES convergence Non-normal matrix Eigenvalues Residual norms 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arioli, M., Pták, V., Strakoš, Z.: Krylov sequences of maximal length and convergence of GMRES. BIT 38(4), 636–643 (1998)CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Baglama, J., Calvetti, D., Golub, G.H., Reichel, L.: Adaptively preconditioned GMRES algorithms. SIAM J. Sci. Comput. 20(1), 243–269 (1998)CrossRefMathSciNetGoogle Scholar
  3. 3.
    Bellalij, M., Jbilou, K., Sadok, H.: New convergence results on the global GMRES method for diagonalizable matrices. J. Comput. Appl. Math. 219, 350–358 (2008)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Campbell, S.L., Ipsen, I.C.F., Kelley, C.T., Meyer, C.D.: GMRES and the minimal polynomial. BIT 36(4), 664–675 (1996)CrossRefMATHMathSciNetGoogle Scholar
  5. 5.
    Carpentieri, B., Duff, I.S., Giraud, L.: A class of spectral two-level preconditioners. SIAM J. Sci. Comput. 25(2), 749–765 (2003)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Carpentieri, B., Giraud, L., Gratton, S.: Additive and multiplicative two-level spectral preconditioning for general linear systems. SIAM J. Sci. Comput. 29(4), 1593–1612 (2007)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Chapman, A., Saad, Y.: Deflated and augmented Krylov subspace techniques. Numer. Linear Algebra Appl. 4(1), 43–66 (1997)CrossRefMATHMathSciNetGoogle Scholar
  8. 8.
    Duintjer Tebbens, J., Meurant, G.: Any Ritz value behavior is possible for Arnoldi and for GMRES. SIAM J. Matrix Anal. Appl. 33(3), 958–978 (2012)CrossRefMATHMathSciNetGoogle Scholar
  9. 9.
    Duintjer Tebbens, J., Meurant, G.: Prescribing the behavior of early terminating GMRES and Arnoldi iterations. Num. Alg. 65(1), 69–90 (2014)CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Duintjer Tebbens, J., Meurant, G., Sadok, H., Strakoš, Z.: On investigating GMRES convergence using unitary matrices. Lin. Alg. Appl. 450, 83–107 (2014)CrossRefMATHGoogle Scholar
  11. 11.
    Eiermann, M.: Fields of values and iterative methods. Lin. Alg. Appl. 180, 167–197 (1993)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Erhel, J., Burrage, K., Pohl, B.: Restarted GMRES preconditioned by deflation. J. Comput. Appl. Math. 69(2), 303–318 (1996)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Gautschi, W.: On inverses of Vandermonde and confluent Vandermonde matrices. III. Numer. Math. 29, 445–450 (1977/78)CrossRefMathSciNetGoogle Scholar
  14. 14.
    Giraud, L., Gratton, S., Martin, E.: Incremental spectral preconditioners for sequences of linear systems. Appl. Numer. Math. 57(11–12), 1164–1180 (2007)CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Giraud, L., Gratton, S., Pinel, X., Vasseur, X.: Flexible GMRES with deflated restarting. SIAM J. Sci. Comput. 32(4), 1858–1878 (2010)CrossRefMATHMathSciNetGoogle Scholar
  16. 16.
    Greenbaum, A.: Generalizations of the field of values useful in the study of polynomial functions of a matrix. Lin. Alg. Appl. 347, 233–249 (2002)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Greenbaum, A., Pták, V., Strakoš, Z.: Any nonincreasing convergence curve is possible for GMRES. SIAM J. Matrix Anal. Appl. 17(3), 465–469 (1996)CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Greenbaum, A., Strakoš, Z.: Matrices that generate the same Krylov residual spaces. In: Recent Advances in Iterative Methods, volume 60 of IMA Vol. Math. Appl., pp. 95–118. Springer, New York (1994)Google Scholar
  19. 19.
    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Research Nat. Bur. Standards 49, 409–436 (1952)CrossRefMATHMathSciNetGoogle Scholar
  20. 20.
    Huhtanen, M., Nevanlinna, O.: Minimal decompositions and iterative methods. Numer. Math. 86(2), 257–281 (2000)CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    Ipsen, I.C.F.: Expressions and bounds for the GMRES residual. BIT 40(3), 524–535 (2000)CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Kharchenko, S.A., Yu. Yeremin, A.: Eigenvalue translation based preconditioners for the GMRES (k) method. Numer. Linear Algebra Appl. 2(1), 51–77 (1995)CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    Kuijlaars, A.B.J.: Convergence analysis of Krylov subspace iterations with methods from potential theory. SIAM Rev. 48(1), 3–40 (2006)CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Le Calvez, C, Molina, B.: Implicitly restarted and deflated GMRES. Numer. Algorithms 21(1–4), 261–285 (1999). Numerical methods for partial differential equations (Marrakech, 1998)CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Liesen, J., Rozložník, M., Strakoš, Z.: Least squares residuals and minimal residual methods. SIAM J. Sci. Comput. 23(5), 1503–1525 (2002)CrossRefMATHMathSciNetGoogle Scholar
  26. 26.
    Liesen, J., Strakoš, Z.: Convergence of GMRES for tridiagonal Toeplitz matrices. SIAM J. Matrix Anal. Appl. 26(1), 233–251 (2004)CrossRefMATHMathSciNetGoogle Scholar
  27. 27.
    Liesen, J., Tichý, P.: Convergence analysis of Krylov subspace methods. GAMM Mitt. Ges. Angew Math. Mech. 27(2), 153–173 (2004)MATHMathSciNetGoogle Scholar
  28. 28.
    Liesen, J., Tichý, P.: The worst-case GMRES for normal matrices. BIT 44(1), 79–98 (2004)CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Loghin, D., Ruiz, D., Touhami, A.: Adaptive preconditioners for nonlinear systems of equations. J. Comput. Appl. Math. 189(1–2), 362–374 (2006)CrossRefMATHMathSciNetGoogle Scholar
  30. 30.
    Morgan, R.B.: A restarted GMRES method augmented with eigenvectors. SIAM J. Matrix Anal. Appl. 16(4), 1154–1171 (1995)CrossRefMATHMathSciNetGoogle Scholar
  31. 31.
    Morgan, R.B.: Implicitly restarted GMRES and Arnoldi methods for nonsymmetric systems of equations. SIAM J. Matrix Anal. Appl. 21(4), 1112–1135 (2000)CrossRefMATHMathSciNetGoogle Scholar
  32. 32.
    Morgan, R.B.: GMRES with deflated restarting. SIAM J. Sci. Comput. 24(1), 20–37 (2002)CrossRefMATHMathSciNetGoogle Scholar
  33. 33.
    Nachtigal, N.M., Reddy, S.C., Trefethen, L.N.: How fast are nonsymmetric matrix iterations? SIAM J. Matrix Anal. Appl. 13(3), 778–795 (1992)CrossRefMATHMathSciNetGoogle Scholar
  34. 34.
    Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8(1), 43–71 (1982)CrossRefMATHMathSciNetGoogle Scholar
  35. 35.
    Parks, M.L., de Sturler, E., Mackey, G., Johnson, D.D., Maiti, S.: Recycling Krylov subspaces for sequences of linear systems. SIAM J. Sci. Comput. 28(5), 1651–1674 (2006)CrossRefMATHMathSciNetGoogle Scholar
  36. 36.
    Pestana, J., Wathen, A.: On choice of preconditioner for minimum residual methods for non-hermitian matrices. J. Comput. Appl. Math. 249, 57–68 (2013)CrossRefMATHMathSciNetGoogle Scholar
  37. 37.
    Saad, Y.: Iterative methods for sparse linear systems. Society for Industrial and Applied Mathematics, 2nd edn. Philadelphia (2003)Google Scholar
  38. 38.
    Saad, Y., Schultz, M.H.: GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J. Sci. Stat. Comput. 7(3), 856–869 (1986)CrossRefMATHMathSciNetGoogle Scholar
  39. 39.
    Sadok, H.: Analysis of the convergence of the minimal and the orthogonal residual methods. Numer. Algorithms 40(2), 201–216 (2005)CrossRefMATHMathSciNetGoogle Scholar
  40. 40.
    Stewart, G.W.: Collinearity and least squares regression. Stat. Sci. 2(1), 68–100 (1987)CrossRefGoogle Scholar
  41. 41.
    Stynes, M.: Steady-state convection-diffusion problems. Acta Numer. 14, 445–508 (2005)CrossRefMATHMathSciNetGoogle Scholar
  42. 42.
    Tichý, P., Liesen, J., Faber, V.: On worst-case GMRES, ideal GMRES, and the polynomial numerical hull of a Jordan block. Electron. Trans. Numer. Anal. 26, 453–473 (2007)MATHMathSciNetGoogle Scholar
  43. 43.
    Titley-Peloquin, D., Pestana, J., Wathen, A.: GMRES convergence bounds that depend on the right-hand side vector. IMA J. Numer. Anal. 34(2), 462–479 (2014)CrossRefMATHMathSciNetGoogle Scholar
  44. 44.
    Trefethen, L.N., Embree, M.: Spectra and Pseudospectra. Princeton University Press, Princeton (2005)MATHGoogle Scholar
  45. 45.
    Zítko, J.: Generalization of convergence conditions for a restarted GMRES. Numer. Linear Algebra Appl. 7, 117–131 (2000)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.ParisFrance
  2. 2.Institute of Computer Science, Academy of Sciences of the Czech RepublicPragueCzech Republic
  3. 3.Faculty of Pharmacy in Hradec KrálovéCharles University in PragueHradec KrálovéCzech Republic

Personalised recommendations