Skip to main content
SpringerLink
Log in

GCV for Tikhonov regularization by partial SVD

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58, 8:1–8:34 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  2. Baglama, J., Fenu, C., Reichel, L., Rodriguez, G.: Analysis of directed networks via partial singular value decomposition and Gauss quadrature. Linear Algebra Appl. 456, 93–121 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  3. Baglama, J., Reichel, L.: Restarted block Lanczos bidiagonalization methods. Numer. Algorithms 43, 251–272 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  4. Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT 53, 285–310 (2013)

    MATH  MathSciNet  Google Scholar 

  5. Bai, Z., Fahey, M., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74, 71–89 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  6. Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 18, 659–670 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  7. Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)

    Book  MATH  Google Scholar 

  8. Brezinski, C., Fika, P., Mitrouli, M.: Estimations of the trace of powers of self-adjoint operators by extrapolation of the moments. Electron. Trans. Numer. Anal. 39, 144–159 (2012)

    MATH  MathSciNet  Google Scholar 

  9. Brezinski, C., Fika, P., Mitrouli, M.: Moments of a linear operator on a Hilbert space, with applications to the trace of the inverse of matrices and the solution of equations. Numer. Linear Algebra Appl. 19, 937–953 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  10. Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1979)

    Article  MATH  Google Scholar 

  11. Eldén, L.: A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT 22, 487–501 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  12. Eldén, L.: A note on the computation of the generalized cross-validation function for ill-conditioned least squares problems. BIT 24, 467–472 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  13. Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)

    Book  MATH  Google Scholar 

  14. Fenu, C., Reichel, L., Rodriguez, G.: GCV for Tikhonov regularization via global Golub–Kahan decomposition. Numer. Linear Algebra Appl. 23, 467–484 (2016)

    Article  MATH  MathSciNet  Google Scholar 

  15. Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)

    MATH  MathSciNet  Google Scholar 

  16. Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  17. Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular Vectors of a matrix. ACM Trans. Math. Software 7, 149–169 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  18. Golub, G.H., Meurant, G.: Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)

    MATH  Google Scholar 

  19. Golub, G.H., von Matt, U.: Generalized cross-validation for large-scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)

    MATH  MathSciNet  Google Scholar 

  20. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)

    Book  Google Scholar 

  21. Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  22. Hochstenbach, M.E.: A Jacobi–Davidson type SVD method. SIAM J. Sci. Comput. 23, 606–628 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  23. Hochstenbach, M.E.: Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems. BIT 44, 721–754 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  24. Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Statist. Simula. 18, 1059–1076 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  25. Jia, Z., Niu, D.: An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J. Matrix Anal. Appl. 25, 246–265 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  26. Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)

    MATH  MathSciNet  Google Scholar 

  27. Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  28. Novati, P., Russo, M.R.: A GCV based Arnoldi–Tikhonov regularization method. BIT 54, 501–521 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  29. Onunwor, E., Reichel, L.: On the computation of a truncated SVD of a large linear discrete ill-posed problem. Numer. Algorithms, in press

  30. Redivo-Zaglia, M., Rodriguez, G.: smt: a matlab toolbox for structured matrices. Numer. Algorithms 59, 639–659 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  31. Stoll, M.: A Krylov–Schur approach to the truncated SVD. Linear Algebra Appl. 436, 2795–2806 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  32. Tang, J., Saad, Y.: A probing method for computing the diagonal of the matrix inverse. Numer. Linear Algebra Appl. 19, 485–501 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  33. van der Mee, C.V.M., Seatzu, S.: A method for generating infinite positive self-adjoint test matrices and Riesz bases. SIAM J. Matrix Anal. Appl. 26, 1132–1149 (2005)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Michiel Hochstenbach and a referee for comments. Work of C.F and G.R. was partially supported by INdAM-GNCS.

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Università di Pisa, Largo Pontecorvo 3, 56127, Pisa, Italy

    Caterina Fenu

  2. Department of Mathematical Sciences, Kent State University, Kent, OH, 44242, USA

    Lothar Reichel

  3. Dipartimento di Matematica e Informatica, Università di Cagliari, viale Merello 92, 09123, Cagliari, Italy

    Giuseppe Rodriguez

  4. Laboratoire de Mathématiques Pures et Appliquées, Université du Littoral, Centre Universtaire de la Mi-Voix, Batiment H. Poincarré, 50 Rue F. Buisson, BP 699, 62228, Calais cedex, France

    Hassane Sadok

Authors
  1. Caterina Fenu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lothar Reichel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Giuseppe Rodriguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hassane Sadok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Caterina Fenu.

Additional information

Communicated by Michiel Hochstenbach.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fenu, C., Reichel, L., Rodriguez , G. et al. GCV for Tikhonov regularization by partial SVD. Bit Numer Math 57, 1019–1039 (2017). https://doi.org/10.1007/s10543-017-0662-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-017-0662-0

Keywords

Mathematics Subject Classification

Search

Navigation