Abstract
The numerical treatment of large-scale discrete ill-posed problems is often accomplished iteratively by projecting the original problem onto a \(k\)-dimensional subspace with \(k\) acting as regularization parameter. Hence, to filter out the contribution of noise in the computed solution, the iterative process must be stopped early. In this paper, we analyze in detail a stopping rule for LSQR proposed recently by the authors, and show how to extend it to Krylov subspace methods such as GMRES, MINRES, etc. Like the original rule, the extended version works well without requiring a priori knowledge about the error norm and stops automatically after \(\widetilde{k}+1\) steps where \(\widetilde{k}\) is the computed regularization parameter. The performance of the stopping rule on several test problems is illustrated numerically.
Similar content being viewed by others
References
Bakushinski AB (1984) Remarks on choosing a regularization parameter using quasi-optimality and ratio criterion. USSR Comp Math Phys 24:181–182
Bauer F, Lukas MA (2011) Comparing parameter choice methods for regularization of ill-posed problem. Math Comput Simul 81:1795–1841
Bazán FSV (2008) Fixed-point iterations in determining the Tikhonov regularization parameter. Inverse Problems 24 doi:10.1088/0266-5611/24/3/035001
Bazán FSV, Francisco JB (2009) An improved fixed-point algorithm for determining the Tikhonov regularization parameter. Inverse Problems 25 doi:10.1088/0266-5611/25/4/045007
Bazán FSV, Cunha MCC, Borges LS (2013) Extension of GKB-FP algorithm to large-scale general-form Tikhonov regularization. Numer Lin Alg. doi:10.1002/nla.1874
Björck Å (1996) Numerical Methods for Least Squares Problems. SIAM
Bunse-Gerstner A, Guerra-Ones V (2006) An improved preconditioned LSQR for discrete ill-posed problems. Math Comput in Simul 73:65–75
Calvetti D, Lewis B, Reichel L (2000) GMRES-type methods for inconsistent systems. Lin Alg and its Appl 316:157–169
Calvetti D, Reichel L, Shuibi A (2005) Invertible smoothing preconditioners for linear discrete ill-posed problems. Appl Num Math 54:135–149
Castellanos JJ, Gómez S, Guerra V (2002) The triangle method for finding the corner of the L-curve. Appl Numer Math 43:359–373
Cimmino G (1983) Calcolo approssimato per le soluzioni dei sistemi di equazioni lineari. La Ricerca Scientifica II 9:326–333
Dold A, Eckmann B (1986) Lecture Notes in Mathematics. Inverse Problems
Eldén L (1982) A weighted pseudoinverse, generalized singular values, and constrained least square problems. BIT 22:487–502
Engl HW, Hanke M, Neubauer A (1996) Mathematics and its applications., Regularization of inverse problemsKluwer Academic, Dordrecht
Fierro RD, Bunch JR (1995) Bounding the subspaces from rank revealing two-sided orthogonal decomposition. SIAM J Matrix Anal Appl 16:743–759
Golub GH, Van Loan CF (1996) Matrix computations. The Johns Hopkins University Press, Baltimore
Hanke H (1995) Conjugate gradient type methods for Ill-posed problems. Longman, Harlow
Hanke M, Hansen PC (1993) Regularization methods for large-scale problems. Surv Math Ind 3:253–315
Hansen PC (1990) The discrete picard condition for discrete Ill-posed problems. BIT 30:658–672
Hansen PC (1994) Regularization tools: a MATLAB package for analysis and solution of discrete ill-posed problems. Numer Algebra 6:1–35
Hansen PC (1998) Rank-deficient and discrete ill-posed problems. SIAM, Philadelphia
Hansen PC (2010) Discrete inverse problems: insight and algorithms. SIAM, Philadelphia
Hansen PC, O’Leary DP (1993) The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput 14:1487–1503
Hansen PC, Jensen TK (2006) Smoothing-norm preconditioning for regularizing minimum-residual methods. SIAM J Matrix Anal Appl 29:1–14
Hansen PC, Jensen TK (2008) Noise propagation in regularizing iterations for image deblurring. Elec Trans Numer Anal 31:204–220
Hansen PC, Jensen TK, Rodriguez G (2007) An adaptive pruning algorithm for the discrete L-curve criterion. J Comput Appl Math 198:483–492
Hestenes MR, Stiefel E (1952) Methods of conjugate gradients for solving linear systems. J Res Nat Bur Stand 49:409–436
Jensen TK, Hansen PC (2007) Iterative regularization with minimum-residual methods. BIT 47:103–120
Kilmer ME, Hansen PC, Español MI (2007) A projection-based approach to general-form Tikhonov regularization. SIAM J Sci Comput 29:315–330
Kilmer ME, O’Leary DP (2001) Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J Matrix Anal Appl 22:1204–1221
Kindermann S (2011) Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron Trans Numer Anal 38:233–257
Landweber L (1951) An iteration formula for Fredholm integral equations of the first kind. Am J Math 73:615–624
Lukas MA (2006) Robust generalized cross-validation for choosing the regularization parameter. Inverse Probl 22:1883–1902
Neuman A, Reichel L, Sadok H (2012) Implementations of range restricted iterative methods for linear discrete ill-posed problems. Linear Algebra Appl 436:3974–3990
Paige CC, Saunders MA (1975) Solution of sparse indefinite systems of linear equations. SIAM J Numer Anal 12:617–629
Morozov VA (1984) Regularization methods for solving incorrectly posed problems. Springer, New York
Paige CC, Saunders MA (1982) LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans Math Softw 8:43–71
Regińska T (1996) A regularization parameter in discrete ill-posed problems. SIAM J Sci Comput 3:740–749
Reichel L, Rodriguez G (2012) Old and new parameter choice rules for discrete ill-posed problems. Numer Algorithms. doi:10.1007/s11075-012-9612-8
Rodriguez G, Theis D (2005) An algorithm for estimating the optimal regularization parameter by the L-curve. Rend Mat 25:69–84
Saad Y, Schultz MH (1986) GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM J Sci Statist Comput 7:856–869
Tikhonov AN (1963) Solution of incorrectly formulated problems and the regularization method. Soviet Math Dokl 4:1035–1038
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Jinyun Yuan.
Leonardo S. Borges Part of this research is supported by FAPESP, Brazil, grant 2009/52193-1.
Fermín S. Viloche Bazán The work of this author is supported by CNPq, Brazil, grants 308709/2011-0, 477093/2011-6.
Rights and permissions
About this article
Cite this article
Borges, L.S., Viloche Bazán, F.S. & Cunha, M.C.C. Automatic stopping rule for iterative methods in discrete ill-posed problems. Comp. Appl. Math. 34, 1175–1197 (2015). https://doi.org/10.1007/s40314-014-0174-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40314-014-0174-3