Approximating the extreme Ritz values and upper bounds for the A-norm of the error in CG

  • Gérard Meurant
  • Petr TichýEmail author
Original Paper


In practical conjugate gradient (CG) computations, it is important to monitor the quality of the approximate solution to Ax = b so that the CG algorithm can be stopped when the required accuracy is reached. The relevant convergence characteristics, like the A-norm of the error or the normwise backward error, cannot be easily computed. However, they can be estimated. Such estimates often depend on approximations of the smallest or largest eigenvalue of A. In the paper, we introduce a new upper bound for the A-norm of the error, which is closely related to the Gauss-Radau upper bound, and discuss the problem of choosing the parameter µ which should represent a lower bound for the smallest eigenvalue of A. The new bound has several practical advantages, the most important one is that it can be used as an approximation to the A-norm of the error even if µ is not exactly a lower bound for the smallest eigenvalue of A. In this case, µ can be chosen, e.g., as the smallest Ritz value or its approximation. We also describe a very cheap algorithm, based on the incremental norm estimation technique, which allows to estimate the smallest and largest Ritz values during the CG computations. An improvement of the accuracy of these estimates of extreme Ritz values is possible, at the cost of storing the CG coefficients and solving a linear system with a tridiagonal matrix at each CG iteration. Finally, we discuss how to cheaply approximate the normwise backward error. The numerical experiments demonstrate the efficiency of the estimates of the extreme Ritz values, and show their practical use in error estimation in CG.


Conjugate gradients Error norm estimation Approximation of Ritz values Incremental norm estimator 

Mathematics Subject Classification (2010)

65F10 65F15 65F35 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



The authors thank an anonymous referee for the very helpful comments.


  1. 1.
    Arioli, M., Duff, I.S., Ruiz, D.: Stopping criteria for iterative solvers. SIAM J. Matrix Anal. Appl. 13(1), 138–144 (1992)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Barrett, R., Berry, M., Chan, T.F., et al.: Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (1994)CrossRefGoogle Scholar
  3. 3.
    Bischof, C.H.: Incremental condition estimation. SIAM J. Matrix Anal. Appl. 11(2), 312–322 (1990)MathSciNetCrossRefGoogle Scholar
  4. 4.
    Dahlquist, G., Eisenstat, S.C., Golub, G.H.: Bounds for the error of linear systems of equations using the theory of moments. J. Math. Anal. Appl. 37, 151–166 (1972)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Dahlquist, G., Golub, G.H., Nash, S.G.: Bounds for the error in linear systems. In: Semi-Infinite Programming (Proc. Workshop, Bad Honnef, 1978), Lecture Notes in Control and Information Sci., vol. 15, pp. 154–172. Springer, Berlin (1979)Google Scholar
  6. 6.
    Duff, I.S., Vömel, C.: Incremental norm estimation for dense and sparse matrices. BIT 42(2), 300–322 (2002)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Duintjer Tebbens, J., Tuma, M.: On incremental condition estimators in the 2-norm. SIAM J. Matrix Anal. Appl. 35(1), 174–197 (2014)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Eiermann, M., Ernst, O.G.: Geometric aspects of the theory of Krylov subspace methods. Acta Numer. 10, 251–312 (2001)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Fischer, B., Golub, G.H.: On the error computation for polynomial based iteration methods. In: Recent Advances in Iterative Methods, IMA Vol. Math. Appl., Vol. 60, pp. 59–67. Springer, New York (1994)Google Scholar
  10. 10.
    Golub, G.H., Meurant, G.: Matrices, moments and quadrature. In: Numerical Analysis 1993 (Dundee, 1993), Pitman Res. Notes Math. Ser., Vol. 303, pp. 105–156. Longman Sci. Tech., Harlow (1994)Google Scholar
  11. 11.
    Golub, G.H., Meurant, G.: Matrices, moments and quadrature. II. How to compute the norm of the error in iterative methods. BIT 37(3), 687–705 (1997)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)zbMATHGoogle Scholar
  13. 13.
    Golub, G.H., Strakoš, Z.: Estimates in quadratic formulas. Numer. Algorithms 8(2-4), 241–268 (1994)MathSciNetCrossRefGoogle Scholar
  14. 14.
    Golub, G.H., Van Loan, C.F.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 4th edn. Johns Hopkins University Press, Baltimore (2013)Google Scholar
  15. 15.
    Greenbaum, A.: Estimating the attainable accuracy of recursively computed residual methods. SIAM J. Matrix Anal. Appl. 18(3), 535–551 (1997)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Gutknecht, M.H., Rozložník, M.: By how much can residual minimization accelerate the convergence of orthogonal residual methods? Numer. Algorithms 27(2), 189–213 (2001)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Gutknecht, M.H., Rozložnik, M.: Residual smoothing techniques: do they improve the limiting accuracy of iterative solvers? BIT 41(1), 86–114 (2001)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–436 (1952)MathSciNetCrossRefGoogle Scholar
  19. 19.
    Higham, N.J.: Accuracy and stability of numerical algorithms. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (1996)zbMATHGoogle Scholar
  20. 20.
    Kouhia, R.: Description of the CYLSHELL set. Laboratory of Structural Mechanics, Finland (1998)Google Scholar
  21. 21.
    Meurant, G.: The computation of bounds for the norm of the error in the conjugate gradient algorithm. Numer. Algorithms 16(1), 77–87 (1998)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Meurant, G.: The Lanczos and Conjugate Gradient Algorithms, from Theory to Finite Precision Computations, Software, Environments and Tools, vol. 19. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2006)zbMATHGoogle Scholar
  23. 23.
    Meurant, G., Tichý, P.: On computing quadrature-based bounds for the A-norm of the error in conjugate gradients. Numer. Algo. 62(2), 163–191 (2013)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Meurant, G., Tichý, P.: Erratum to: On computing quadrature-based bounds for the A-norm of the error in conjugate gradients [mr3011386]. Numer. Algorithms 66(3), 679–680 (2014)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Oettli, W., Prager, W.: Compatibility of approximate solution of linear equations with given error bounds for coefficients and right-hand sides. Numer. Math. 6(1), 405–409 (1964)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Software 8(1), 43–71 (1982)MathSciNetCrossRefGoogle Scholar
  27. 27.
    Parlett, B.N., Dhillon, I.S.: Relatively robust representations of symmetric tridiagonals. In: Proceedings of the International Workshop on Accurate Solution of Eigenvalue Problems (University Park, PA, 1998), vol. 309, pp. 121–151 (2000)Google Scholar
  28. 28.
    Rigal, J.L., Gaches, J.: On the compatibility of a given solution with the data of a linear system. Journal of the ACM (JACM) 14(3), 543–548 (1967)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Strakoš, Z., Tichý, P.: On error estimation in the conjugate gradient method and why it works in finite precision computations. Electron. Trans. Numer. Anal. 13, 56–80 (2002)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Strakoš, Z., Tichý, P.: Error estimation in preconditioned conjugate gradients. BIT 45(4), 789–817 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.ParisFrance
  2. 2.Faculty of Mathematics and PhysicsCharles UniversityPrague 8Czech Republic

Personalised recommendations