BIT Numerical Mathematics

, Volume 57, Issue 4, pp 1019–1039 | Cite as

GCV for Tikhonov regularization by partial SVD

  • Caterina Fenu
  • Lothar Reichel
  • Giuseppe Rodriguez
  • Hassane Sadok


Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.


Generalized cross validation Tikhonov regularization Partial singular value decomposition 

Mathematics Subject Classification

65F22 65F20 65R32 



The authors would like to thank Michiel Hochstenbach and a referee for comments. Work of C.F and G.R. was partially supported by INdAM-GNCS.


  1. 1.
    Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58, 8:1–8:34 (2011)CrossRefMATHMathSciNetGoogle Scholar
  2. 2.
    Baglama, J., Fenu, C., Reichel, L., Rodriguez, G.: Analysis of directed networks via partial singular value decomposition and Gauss quadrature. Linear Algebra Appl. 456, 93–121 (2014)CrossRefMATHMathSciNetGoogle Scholar
  3. 3.
    Baglama, J., Reichel, L.: Restarted block Lanczos bidiagonalization methods. Numer. Algorithms 43, 251–272 (2006)CrossRefMATHMathSciNetGoogle Scholar
  4. 4.
    Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT 53, 285–310 (2013)MATHMathSciNetGoogle Scholar
  5. 5.
    Bai, Z., Fahey, M., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74, 71–89 (1996)CrossRefMATHMathSciNetGoogle Scholar
  6. 6.
    Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 18, 659–670 (1988)CrossRefMATHMathSciNetGoogle Scholar
  7. 7.
    Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefMATHGoogle Scholar
  8. 8.
    Brezinski, C., Fika, P., Mitrouli, M.: Estimations of the trace of powers of self-adjoint operators by extrapolation of the moments. Electron. Trans. Numer. Anal. 39, 144–159 (2012)MATHMathSciNetGoogle Scholar
  9. 9.
    Brezinski, C., Fika, P., Mitrouli, M.: Moments of a linear operator on a Hilbert space, with applications to the trace of the inverse of matrices and the solution of equations. Numer. Linear Algebra Appl. 19, 937–953 (2012)CrossRefMATHMathSciNetGoogle Scholar
  10. 10.
    Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1979)CrossRefMATHGoogle Scholar
  11. 11.
    Eldén, L.: A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT 22, 487–501 (1982)CrossRefMATHMathSciNetGoogle Scholar
  12. 12.
    Eldén, L.: A note on the computation of the generalized cross-validation function for ill-conditioned least squares problems. BIT 24, 467–472 (1984)CrossRefMATHMathSciNetGoogle Scholar
  13. 13.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)CrossRefMATHGoogle Scholar
  14. 14.
    Fenu, C., Reichel, L., Rodriguez, G.: GCV for Tikhonov regularization via global Golub–Kahan decomposition. Numer. Linear Algebra Appl. 23, 467–484 (2016)CrossRefMATHMathSciNetGoogle Scholar
  15. 15.
    Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)MATHMathSciNetGoogle Scholar
  16. 16.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)CrossRefMATHMathSciNetGoogle Scholar
  17. 17.
    Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular Vectors of a matrix. ACM Trans. Math. Software 7, 149–169 (1981)CrossRefMATHMathSciNetGoogle Scholar
  18. 18.
    Golub, G.H., Meurant, G.: Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)MATHGoogle Scholar
  19. 19.
    Golub, G.H., von Matt, U.: Generalized cross-validation for large-scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)MATHMathSciNetGoogle Scholar
  20. 20.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)CrossRefGoogle Scholar
  21. 21.
    Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)CrossRefMATHMathSciNetGoogle Scholar
  22. 22.
    Hochstenbach, M.E.: A Jacobi–Davidson type SVD method. SIAM J. Sci. Comput. 23, 606–628 (2001)CrossRefMATHMathSciNetGoogle Scholar
  23. 23.
    Hochstenbach, M.E.: Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems. BIT 44, 721–754 (2004)CrossRefMATHMathSciNetGoogle Scholar
  24. 24.
    Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Statist. Simula. 18, 1059–1076 (1989)CrossRefMATHMathSciNetGoogle Scholar
  25. 25.
    Jia, Z., Niu, D.: An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J. Matrix Anal. Appl. 25, 246–265 (2003)CrossRefMATHMathSciNetGoogle Scholar
  26. 26.
    Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)MATHMathSciNetGoogle Scholar
  27. 27.
    Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)CrossRefMATHMathSciNetGoogle Scholar
  28. 28.
    Novati, P., Russo, M.R.: A GCV based Arnoldi–Tikhonov regularization method. BIT 54, 501–521 (2014)CrossRefMATHMathSciNetGoogle Scholar
  29. 29.
    Onunwor, E., Reichel, L.: On the computation of a truncated SVD of a large linear discrete ill-posed problem. Numer. Algorithms, in pressGoogle Scholar
  30. 30.
    Redivo-Zaglia, M., Rodriguez, G.: smt: a matlab toolbox for structured matrices. Numer. Algorithms 59, 639–659 (2012)CrossRefMATHMathSciNetGoogle Scholar
  31. 31.
    Stoll, M.: A Krylov–Schur approach to the truncated SVD. Linear Algebra Appl. 436, 2795–2806 (2012)CrossRefMATHMathSciNetGoogle Scholar
  32. 32.
    Tang, J., Saad, Y.: A probing method for computing the diagonal of the matrix inverse. Numer. Linear Algebra Appl. 19, 485–501 (2012)CrossRefMATHMathSciNetGoogle Scholar
  33. 33.
    van der Mee, C.V.M., Seatzu, S.: A method for generating infinite positive self-adjoint test matrices and Riesz bases. SIAM J. Matrix Anal. Appl. 26, 1132–1149 (2005)CrossRefMATHMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  1. 1.Dipartimento di InformaticaUniversità di PisaPisaItaly
  2. 2.Department of Mathematical SciencesKent State UniversityKentUSA
  3. 3.Dipartimento di Matematica e InformaticaUniversità di CagliariCagliariItaly
  4. 4.Laboratoire de Mathématiques Pures et AppliquéesUniversité du Littoral, Centre Universtaire de la Mi-VoixCalais cedexFrance

Personalised recommendations