Skip to main content
Log in

On the choice of subspace for large-scale Tikhonov regularization problems in general form

  • Original Paper
  • Published:
Numerical Algorithms Aims and scope Submit manuscript

Abstract

Many applications in science and engineering require the solution of large linear discrete ill-posed problems. The matrices that define these problems are very ill-conditioned and possibly numerically singular, and the right-hand sides, which represent the measured data, typically are contaminated by measurement error. Straightforward solution of these problems generally is not meaningful due to severe error propagation. Tikhonov regularization seeks to alleviate this difficulty by replacing the given linear discrete ill-posed problem by a penalized least-squares problem, whose solution is less sensitive to the error in the right-hand side. The penalty term is determined by a regularization matrix. A suitable choice of this matrix may result in a computed solution of higher quality than when the regularization matrix is the identity. Two iterative solution methods based on the global Arnoldi decomposition method have been proposed in the literature for the solution of large-scale penalized least-squares problems that stem from Tikhonov regularization. In one of these, the regularization matrix influences the choice of the solution subspace; in the other one, it does not. This paper compares these approaches both with respect to the quality of the computed solution and computing time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Bentbib, A.H., El Guide, M., Jbilou, K., Reichel, L.: Global Golub–Kahan bidiagonalization for image restoration. J. Comput. Appl. Math. 300, 233–244 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  2. Bouhamidi, A., Jbilou, K.: Sylvester Tikhonov-regularization methods in image restoration. J. Comput. Appl. Math. 206, 86–98 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bouhamidi, A., Jbilou, K., Reichel, L., Sadok, H.: A generalized global Arnoldi method for ill-posed matrix equations. J. Comput. Appl. Math. 236, 2078–2089 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. Brezinski, C., Redivo–Zaglia, M., Rodriguez, G., Seatzu, S.: Extrapolation techniques for ill-conditioned linear systems. Numer. Math. 81, 1–29 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  5. Brezinski, C., Rodriguez, G., Seatzu, S.: Error estimates for the regularization of least squares problems. Numer. Algorithms 51, 61–76 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  6. Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large, discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  7. Calvetti, D., Reichel, L., Shuibi, A.: Invertible smoothing preconditioners for linear discrete ill-posed problems. Appl. Numer. Math. 54, 135–149 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  8. Daniel, J.W., Gragg, W.B., Kaufman, L., Stewart, G.W.: Reorthogonalization and stable algorithms for updating the Gram-Schmidt QR factorization. Math. Comp. 30, 772–795 (1976)

    MathSciNet  MATH  Google Scholar 

  9. Donatelli, M., Reichel, L.: Square smoothing regularization matrices with accurate boundary conditions. J. Comput. Appl. Math. 272, 334–349 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  10. Donatelli, M., Neuman, A., Reichel, L.: Square regularization matrices for large linear discrete ill-posed problems. Numer. Linear Algebra Appl. 19, 896–913 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Dykes, L., Huang, G., Noschese, S., Reichel, L.: Regularization matrices for discrete ill-posed problems in several space-dimensions. Numerical Linear Algebra Appl. (2018, in press)

  12. Elbouyahyaout, L., Messaoudi, A., Sadok, H.: Algebraic properties of the block GMRES and block Arnoldi methods. Electron. Trans. Numer. Anal. 33, 207–220 (2009)

    MathSciNet  MATH  Google Scholar 

  13. Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)

    Book  MATH  Google Scholar 

  14. Frommer, A., Lund, K., Szyld, D.B.: Block Krylov subspace methods for functions of matrices. Electron. Trans. Numer. Anal. 47, 100–126 (2017)

    MathSciNet  MATH  Google Scholar 

  15. Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)

    MathSciNet  MATH  Google Scholar 

  16. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)

    Book  Google Scholar 

  17. Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hochstenbach, M.E., Reichel, L.: An iterative method for Tikhonov regularization with a general linear regularization operator. J. Integral Equations Appl. 22, 463–480 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  19. Horn, R.A., Johnson, C.R.: Topics in Matrix Analysis. Cambridge University Press, Cambridge (1991)

    Book  MATH  Google Scholar 

  20. Huang, G., Noschese, S., Reichel, L.: Regularization matrices determined by matrix nearness problems. Linear Algebra Appl. 502, 41–57 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  21. Jbilou, K., Messaoudi, A., Sadok, H.: Global FOM and GMRES algorithms for matrix equations. Appl. Numer. Math. 31, 49–63 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  22. Jbilou, K., Sadok, H., Tinzefte, A.: Oblique projection methods for linear systems with multiple right-hand sides. Electron. Trans. Numer. Anal. 20, 119–138 (2005)

    MathSciNet  MATH  Google Scholar 

  23. Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general-form Tikhonov regularization. SIAM J. Sci. Comput. 29, 315–330 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)

    MathSciNet  MATH  Google Scholar 

  25. Lampe, J., Reichel, L., Voss, H.: Large-scale Tikhonov regularization via reduction by orthogonal projection. Linear Algebra Appl. 436, 2845–2865 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lewis, B., Reichel, L.: Arnoldi–Tikhonov regularization methods. J. Comput. Appl. Math. 226, 92–102 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  27. Li, R.C, Ye, Q.: A Krylov subspace method for quadratic matrix polynomials with application to constrained least squares problems. SIAM J. Matrix Anal. Appl. 25, 405–428 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  28. Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  29. Noschese, S., Reichel, L.: Inverse problems for regularization matrices. Numer. Algorithms 60, 531–544 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  30. Reichel, L., Rodriguez, G.: Old and new parameter choice rules for discrete ill-posed problems. Numer. Algorithms 63, 65–87 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  31. Reichel, L., Yu, X.: Matrix decompositions for Tikhonov regularization. Electron. Trans. Numer. Anal. 43, 223–243 (2015)

    MathSciNet  MATH  Google Scholar 

  32. Reichel, L., Yu, X.: Tikhonov regularization via flexible Arnoldi reduction. BIT 55, 1145–1168 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  33. Reichel, L., Rodriguez, G., Seatzu, S.: Error estimates for large-scale ill-posed problems. Numer. Algorithms 51, 341–361 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  34. Reichel, L., Sgallari, F., Ye, Q.: Tikhonov regularization based on generalized Krylov subspace methods. Appl. Numer. Math. 62, 1215–1228 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  35. Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  Google Scholar 

Download references

Funding

Research by GH was supported in part by the Fund of Application Foundation of Science and Technology Department of the Sichuan Province (2016JY0249) and by NNSF (2017YFC0601505, 41672325), research by LR was supported in part by NSF grants DMS-1729509 and DMS-1720259, and research by FY was supported in part by NNSF (11501392).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Lothar Reichel.

Additional information

In Memory of Sebastiano Seatzu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, G., Reichel, L. & Yin, F. On the choice of subspace for large-scale Tikhonov regularization problems in general form. Numer Algor 81, 33–55 (2019). https://doi.org/10.1007/s11075-018-0534-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11075-018-0534-y

Keywords

Mathematics Subject Classification (2010)

Navigation