Abstract
This paper proposes a new approach for choosing the regularization parameters in multi-parameter regularization methods when applied to approximate the solution of linear discrete ill-posed problems. We consider both direct methods, such as Tikhonov regularization with two or more regularization terms, and iterative methods based on the projection of a Tikhonov-regularized problem onto Krylov subspaces of increasing dimension. The latter methods regularize by choosing appropriate regularization terms and the dimension of the Krylov subspace. Our investigation focuses on selecting a proper set of regularization parameters that satisfies the discrepancy principle and maximizes a suitable quantity, whose size reflects the quality of the computed approximate solution. Theoretical results are shown and illustrated by numerical experiments.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belge, M., Kilmer, M.E., Miller, E.L.: Efficient determination of multiple regularization parameters in a generalized L-curve framework. Inverse Probl. 18, 1161–1183 (2002)
Brezinski, C., Redivo-Zaglia, M., Rodriguez, G., Seatzu, S.: Multi-parameter regularization techniques for ill-conditioned linear systems. Numer. Math. 94, 203–228 (2003)
Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Fornasier, M., Naumova, V., Pereverzyev, S.V.: Parameter choice strategies for multipenalty regularization. SIAM J. Numer. Anal. 52, 1770–1794 (2014)
Gazzola, S., Novati, P.: Multi-parameter Arnoldi–Tikhonov methods. Electron. Trans. Numer. Anal. 40, 452–475 (2013)
Gazzola, S., Novati, P., Russo, M.R.: Embedded techniques for choosing the parameter in Tikhonov regularization. Numer. Linear Algebra Appl. 21, 796–812 (2014)
Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)
Hansen, P.C.: Regularization tools: a matlab package for analysis and solution of discrete ill-posed problems. Numer. Algorithms 6, 1–35 (1994)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)
Hansen, P.C., Nagy, J.G., O’Leary, D.P.: Deblurring Images: Matrices, Spectra and Filtering. SIAM, Philadelphia (2006)
Hochstenbach, M.E., McNinch, N., Reichel, L.: Discrete ill-posed least-squares problems with a solution norm constraint. Linear Algebra Appl. 436, 3801–3818 (2012)
Hochstenbach, M.E., Reichel, L.: An iterative method for Tikhonov regularization with a general linear regularization operator. J. Integral Equ. Appl. 22, 463–480 (2010)
Hochstenbach, M.E., Reichel, L.: Combining approximate solutions for linear discrete ill-posed problems. J. Comput. Appl. Math. 236, 2179–2185 (2012)
Ito, K., Jin, B., Takeuchi, T.: Multi-parameter Tikhonov regularization. Methods Appl. Anal. 18, 31–46 (2011)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)
Kunisch, K., Pock, T.: A bilevel optimization approach for parameter learning in variational methods. SIAM J. Imaging Sci. 6, 938–983 (2013)
Lewis, B., Reichel, L.: Arnoldi–Tikhonov regularization methods. J. Comput. Appl. Math. 226, 92–102 (2009)
Lu, S., Pereverzev, S.V.: Multi-parameter regularization and its numerical realization. Numer. Math. 118, 1–31 (2011)
Lu, S., Pereverzev, S.V., Shao, Y., Tautenhahn, U.: Discrepancy curves for multi-parameter regularization. J. Inverse Ill-Posed Probl. 18, 655–676 (2010)
Mead, J.L.: Discontinuous parameter estimates with least squares estimators. Appl. Math. Comput. 219, 5210–5223 (2013)
Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)
Morozov, V.A.: On the solution of functional equations by the method of regularization. Soviet Math. Dokl. 7, 414–417 (1966)
Nagy, J.G., Palmer, K.M., Perrone, L.: Iterative methods for image deblurring: a matlab object oriented approach. Numer. Algorithms 36, 73–93 (2004). http://www.mathcs.emory.edu/~nagy/RestoreTools
Regińska, T.: Two-parameter discrepancy principle for combined projection and Tikhonov regularization of ill-posed problems. J. Inverse Ill-Posed Probl. 21, 561–577 (2013)
Reichel, L., Sgallari, F., Ye, Q.: Tikhonov regularization based on generalized Krylov subspace methods. Appl. Numer. Math. 62, 1215–1228 (2012)
Reichel, L., Ye, Q.: Simple square smoothing regularization operators. Electron. Trans. Numer. Anal. 33, 63–83 (2009)
Reichel, L., Yu, X.: Matrix decompositions for Tikhonov regularization. Electron. Trans. Numer. Anal. 43, 223–243 (2015)
Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)
Acknowledgments
We are grateful to Giuseppe Rodriguez for providing us with the GCV-based multi-parameter regularization software, as well as to the editor and referees for remarks that lead to improvements of the presentation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Rosemary Renaut.
Research supported in part by the GNCS project “Giovani Ricercatori”, by the “ex-60 %” funds of the University of Padova, and by NSF Grant DMS-1115385.
Rights and permissions
About this article
Cite this article
Gazzola, S., Reichel, L. A new framework for multi-parameter regularization. Bit Numer Math 56, 919–949 (2016). https://doi.org/10.1007/s10543-015-0595-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10543-015-0595-4