Skip to main content
Log in

Automatic stopping rule for iterative methods in discrete ill-posed problems

  • Published:
Computational and Applied Mathematics Aims and scope Submit manuscript

Abstract

The numerical treatment of large-scale discrete ill-posed problems is often accomplished iteratively by projecting the original problem onto a \(k\)-dimensional subspace with \(k\) acting as regularization parameter. Hence, to filter out the contribution of noise in the computed solution, the iterative process must be stopped early. In this paper, we analyze in detail a stopping rule for LSQR proposed recently by the authors, and show how to extend it to Krylov subspace methods such as GMRES, MINRES, etc. Like the original rule, the extended version works well without requiring a priori knowledge about the error norm and stops automatically after \(\widetilde{k}+1\) steps where \(\widetilde{k}\) is the computed regularization parameter. The performance of the stopping rule on several test problems is illustrated numerically.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Bakushinski AB (1984) Remarks on choosing a regularization parameter using quasi-optimality and ratio criterion. USSR Comp Math Phys 24:181–182

    Article  Google Scholar 

  • Bauer F, Lukas MA (2011) Comparing parameter choice methods for regularization of ill-posed problem. Math Comput Simul 81:1795–1841

    Article  MATH  MathSciNet  Google Scholar 

  • Bazán FSV (2008) Fixed-point iterations in determining the Tikhonov regularization parameter. Inverse Problems 24 doi:10.1088/0266-5611/24/3/035001

  • Bazán FSV, Francisco JB (2009) An improved fixed-point algorithm for determining the Tikhonov regularization parameter. Inverse Problems 25 doi:10.1088/0266-5611/25/4/045007

  • Bazán FSV, Cunha MCC, Borges LS (2013) Extension of GKB-FP algorithm to large-scale general-form Tikhonov regularization. Numer Lin Alg. doi:10.1002/nla.1874

  • Björck Å (1996) Numerical Methods for Least Squares Problems. SIAM

  • Bunse-Gerstner A, Guerra-Ones V (2006) An improved preconditioned LSQR for discrete ill-posed problems. Math Comput in Simul 73:65–75

    Article  MATH  MathSciNet  Google Scholar 

  • Calvetti D, Lewis B, Reichel L (2000) GMRES-type methods for inconsistent systems. Lin Alg and its Appl 316:157–169

    Article  MATH  MathSciNet  Google Scholar 

  • Calvetti D, Reichel L, Shuibi A (2005) Invertible smoothing preconditioners for linear discrete ill-posed problems. Appl Num Math 54:135–149

    Article  MATH  MathSciNet  Google Scholar 

  • Castellanos JJ, Gómez S, Guerra V (2002) The triangle method for finding the corner of the L-curve. Appl Numer Math 43:359–373

    Article  MATH  MathSciNet  Google Scholar 

  • Cimmino G (1983) Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. La Ricerca Scientifica II 9:326–333

    Google Scholar 

  • Dold A, Eckmann B (1986) Lecture Notes in Mathematics. Inverse Problems

  • Eldén L (1982) A weighted pseudoinverse, generalized singular values, and constrained least square problems. BIT 22:487–502

    Article  MATH  MathSciNet  Google Scholar 

  • Engl HW, Hanke M, Neubauer A (1996) Mathematics and its applications., Regularization of inverse problemsKluwer Academic, Dordrecht

    Google Scholar 

  • Fierro RD, Bunch JR (1995) Bounding the subspaces from rank revealing two-sided orthogonal decomposition. SIAM J Matrix Anal Appl 16:743–759

    Article  MATH  MathSciNet  Google Scholar 

  • Golub GH, Van Loan CF (1996) Matrix computations. The Johns Hopkins University Press, Baltimore

    MATH  Google Scholar 

  • Hanke H (1995) Conjugate gradient type methods for Ill-posed problems. Longman, Harlow

    MATH  Google Scholar 

  • Hanke M, Hansen PC (1993) Regularization methods for large-scale problems. Surv Math Ind 3:253–315

    MATH  MathSciNet  Google Scholar 

  • Hansen PC (1990) The discrete picard condition for discrete Ill-posed problems. BIT 30:658–672

    Article  MATH  MathSciNet  Google Scholar 

  • Hansen PC (1994) Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer Algebra 6:1–35

    Article  MATH  Google Scholar 

  • Hansen PC (1998) Rank-deficient and discrete ill-posed problems. SIAM, Philadelphia

    Book  Google Scholar 

  • Hansen PC (2010) Discrete inverse problems: insight and algorithms. SIAM, Philadelphia

    Book  Google Scholar 

  • Hansen PC, O’Leary DP (1993) The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14:1487–1503

    Article  MATH  MathSciNet  Google Scholar 

  • Hansen PC, Jensen TK (2006) Smoothing-norm preconditioning for regularizing minimum-residual methods. SIAM J Matrix Anal Appl 29:1–14

    Article  MATH  MathSciNet  Google Scholar 

  • Hansen PC, Jensen TK (2008) Noise propagation in regularizing iterations for image deblurring. Elec Trans Numer Anal 31:204–220

    MATH  MathSciNet  Google Scholar 

  • Hansen PC, Jensen TK, Rodriguez G (2007) An adaptive pruning algorithm for the discrete L-curve criterion. J Comput Appl Math 198:483–492

    Article  MATH  MathSciNet  Google Scholar 

  • Hestenes MR, Stiefel E (1952) Methods of conjugate gradients for solving linear systems. J Res Nat Bur Stand 49:409–436

    Article  MATH  MathSciNet  Google Scholar 

  • Jensen TK, Hansen PC (2007) Iterative regularization with minimum-residual methods. BIT 47:103–120

    Article  MATH  MathSciNet  Google Scholar 

  • Kilmer ME, Hansen PC, Español MI (2007) A projection-based approach to general-form Tikhonov regularization. SIAM J Sci Comput 29:315–330

    Article  MATH  MathSciNet  Google Scholar 

  • Kilmer ME, O’Leary DP (2001) Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J Matrix Anal Appl 22:1204–1221

    Article  MATH  MathSciNet  Google Scholar 

  • Kindermann S (2011) Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron Trans Numer Anal 38:233–257

    MATH  MathSciNet  Google Scholar 

  • Landweber L (1951) An iteration formula for Fredholm integral equations of the first kind. Am J Math 73:615–624

    Article  MATH  MathSciNet  Google Scholar 

  • Lukas MA (2006) Robust generalized cross-validation for choosing the regularization parameter. Inverse Probl 22:1883–1902

    Article  MATH  MathSciNet  Google Scholar 

  • Neuman A, Reichel L, Sadok H (2012) Implementations of range restricted iterative methods for linear discrete ill-posed problems. Linear Algebra Appl 436:3974–3990

    Article  MATH  MathSciNet  Google Scholar 

  • Paige CC, Saunders MA (1975) Solution of sparse indefinite systems of linear equations. SIAM J Numer Anal 12:617–629

    Article  MATH  MathSciNet  Google Scholar 

  • Morozov VA (1984) Regularization methods for solving incorrectly posed problems. Springer, New York

    Book  Google Scholar 

  • Paige CC, Saunders MA (1982) LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8:43–71

  • Regińska T (1996) A regularization parameter in discrete ill-posed problems. SIAM J Sci Comput 3:740–749

    Article  Google Scholar 

  • Reichel L, Rodriguez G (2012) Old and new parameter choice rules for discrete ill-posed problems. Numer Algorithms. doi:10.1007/s11075-012-9612-8

  • Rodriguez G, Theis D (2005) An algorithm for estimating the optimal regularization parameter by the L-curve. Rend Mat 25:69–84

    MATH  MathSciNet  Google Scholar 

  • Saad Y, Schultz MH (1986) GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J Sci Statist Comput 7:856–869

    Article  MATH  MathSciNet  Google Scholar 

  • Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Math Dokl 4:1035–1038

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fermín S. Viloche Bazán.

Additional information

Communicated by Jinyun Yuan.

Leonardo S. Borges Part of this research is supported by FAPESP, Brazil, grant 2009/52193-1.

Fermín S. Viloche Bazán The work of this author is supported by CNPq, Brazil, grants 308709/2011-0, 477093/2011-6.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borges, L.S., Viloche Bazán, F.S. & Cunha, M.C.C. Automatic stopping rule for iterative methods in discrete ill-posed problems. Comp. Appl. Math. 34, 1175–1197 (2015). https://doi.org/10.1007/s40314-014-0174-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40314-014-0174-3

Keywords

Mathematics Subject Classification

Navigation