BIT Numerical Mathematics

, Volume 47, Issue 1, pp 103–120 | Cite as

Iterative regularization with minimum-residual methods

  • T.K. Jensen
  • P.C. Hansen


We study the regularization properties of iterative minimum-residual methods applied to discrete ill-posed problems. In these methods, the projection onto the underlying Krylov subspace acts as a regularizer, and the emphasis of this work is on the role played by the basis vectors of these Krylov subspaces. We provide a combination of theory and numerical examples, and our analysis confirms the experience that MINRES and MR-II can work as general regularization methods. We also demonstrate theoretically and experimentally that the same is not true, in general, for GMRES and RRGMRES – their success as regularization methods is highly problem dependent.

Key words

iterative regularization discrete ill-posed problems GMRES RRGMRES MINRES MR-II Krylov subspaces 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    P. N. Brown and H. F. Walker, GMRES on (nearly) singular systems, SIAM J. Matrix Anal. Appl., 18 (1997), pp. 37–51.zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    D. Calvetti, G. Landi, L. Reichel, and F. Sgallari, Non-negativity and iterative methods for ill-posed problems, Inverse Probl., 20 (2004), pp. 1747–1758.zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    D. Calvetti, B. Lewis, and L. Reichel, GMRES-type methods for inconsistent systems, Linear Algebra Appl., 316 (2000), pp. 157–169.zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    D. Calvetti, B. Lewis, and L. Reichel, GMRES, L-curves, and discrete ill-posed problems, BIT, 42 (2002), pp. 44–65.zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    D. Calvetti, B. Lewis, and L. Reichel, On the regularizing properties of the GMRES method, Numer. Math., 91 (2002), pp. 605–625.zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    B. Fischer, Polynomial Based Iteration Methods for Symmetric Linear Systems, Wiley Teubner, Stuttgart, 1996.zbMATHGoogle Scholar
  7. 7.
    B. Fischer, M. Hanke, and M. Hochbruck, A note on conjugate-gradient type methods for indefinite and/or inconsistent linear systems, Numer. Algorithms, 11 (1996), pp. 181–187.zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    M. Hanke, Conjugate Gradient Type Methods for Ill-Posed Problems, Longman Scientific & Technical, Essex, 1995.zbMATHGoogle Scholar
  9. 9.
    M. Hanke, On Lanczos based methods for the regularization of discrete ill-posed problems, BIT, 41 (2001), pp. 1008–1018.CrossRefMathSciNetGoogle Scholar
  10. 10.
    M. Hanke and J. G. Nagy, Restoration of atmospherically blurred images by symmetric indefinite conjugate gradient techniques, Inverse Probl., 12 (1996), pp. 157–173.zbMATHCrossRefMathSciNetGoogle Scholar
  11. 11.
    P. C. Hansen, Regularization Tools. A Matlab package for analysis and solution of discrete ill-posed problems, Numer. Algorithms, 6 (2004), pp. 1–35.CrossRefGoogle Scholar
  12. 12.
    P. C. Hansen and T. K. Jensen, Smoothing-norm preconditioning for regularizing minimum-residual methods, SIAM J. Matrix Anal. Appl., 29 (2006), pp 1–14.Google Scholar
  13. 13.
    P. C. Hansen, J. G. Nagy, and D. P. O’Leary, Deblurring Images: Matrices, Spectra, and Filtering, Fundamentals of Algorithms 3, SIAM, Philadephia, 2006.Google Scholar
  14. 14.
    R. A. Horn and C. R. Johnson, Matrix Analysis, Cambridge University Press, Cambridge 1985.Google Scholar
  15. 15.
    M. E. Kilmer, On the regularizing properties of Krylov subspace methods, unpublished; results presented at BIT 40th Anniversary meeting, Lund, Sweden, 2000.Google Scholar
  16. 16.
    M. E. Kilmer and G. W. Stewart, Iterative regularization and MINRES, SIAM J. Matrix Anal. Appl., 21 (1999), pp. 613–628.CrossRefMathSciNetGoogle Scholar
  17. 17.
    C. C. Paige and M. A. Saunders, Solution of sparse indefinite systems of linear equations, SIAM J. Numer. Anal., 12 (1975), pp. 617–629.zbMATHCrossRefMathSciNetGoogle Scholar
  18. 18.
    Y. Saad and M. H. Schultz, GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems, SIAM J. Sci. Stat. Comput., 7 (1986), pp. 856–869.zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science + Business Media B.V. 2007

Authors and Affiliations

  1. 1.TNM ConsultHerlevDenmark
  2. 2.Informatics and Mathematical ModellingTechnical University of DenmarkLyngbyDenmark

Personalised recommendations