GCV for Tikhonov regularization by partial SVD

  • Caterina Fenu
  • Lothar Reichel
  • Giuseppe Rodriguez
  • Hassane Sadok


Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.


Generalized cross validation Tikhonov regularization Partial singular value decomposition 

Mathematics Subject Classification

65F22 65F20 65R32 


  1. 1.
    Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58, 8:1–8:34 (2011)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Baglama, J., Fenu, C., Reichel, L., Rodriguez, G.: Analysis of directed networks via partial singular value decomposition and Gauss quadrature. Linear Algebra Appl. 456, 93–121 (2014)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Baglama, J., Reichel, L.: Restarted block Lanczos bidiagonalization methods. Numer. Algorithms 43, 251–272 (2006)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT 53, 285–310 (2013)MathSciNetMATHGoogle Scholar
  5. 5.
    Bai, Z., Fahey, M., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74, 71–89 (1996)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 18, 659–670 (1988)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)CrossRefMATHGoogle Scholar
  8. 8.
    Brezinski, C., Fika, P., Mitrouli, M.: Estimations of the trace of powers of self-adjoint operators by extrapolation of the moments. Electron. Trans. Numer. Anal. 39, 144–159 (2012)MathSciNetMATHGoogle Scholar
  9. 9.
    Brezinski, C., Fika, P., Mitrouli, M.: Moments of a linear operator on a Hilbert space, with applications to the trace of the inverse of matrices and the solution of equations. Numer. Linear Algebra Appl. 19, 937–953 (2012)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1979)CrossRefMATHGoogle Scholar
  11. 11.
    Eldén, L.: A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT 22, 487–501 (1982)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Eldén, L.: A note on the computation of the generalized cross-validation function for ill-conditioned least squares problems. BIT 24, 467–472 (1984)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)CrossRefMATHGoogle Scholar
  14. 14.
    Fenu, C., Reichel, L., Rodriguez, G.: GCV for Tikhonov regularization via global Golub–Kahan decomposition. Numer. Linear Algebra Appl. 23, 467–484 (2016)MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)MathSciNetMATHGoogle Scholar
  16. 16.
    Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular Vectors of a matrix. ACM Trans. Math. Software 7, 149–169 (1981)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Golub, G.H., Meurant, G.: Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)MATHGoogle Scholar
  19. 19.
    Golub, G.H., von Matt, U.: Generalized cross-validation for large-scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)MathSciNetMATHGoogle Scholar
  20. 20.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)CrossRefGoogle Scholar
  21. 21.
    Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Hochstenbach, M.E.: A Jacobi–Davidson type SVD method. SIAM J. Sci. Comput. 23, 606–628 (2001)MathSciNetCrossRefMATHGoogle Scholar
  23. 23.
    Hochstenbach, M.E.: Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems. BIT 44, 721–754 (2004)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Statist. Simula. 18, 1059–1076 (1989)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Jia, Z., Niu, D.: An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J. Matrix Anal. Appl. 25, 246–265 (2003)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)MathSciNetMATHGoogle Scholar
  27. 27.
    Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Novati, P., Russo, M.R.: A GCV based Arnoldi–Tikhonov regularization method. BIT 54, 501–521 (2014)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Onunwor, E., Reichel, L.: On the computation of a truncated SVD of a large linear discrete ill-posed problem. Numer. Algorithms, in pressGoogle Scholar
  30. 30.
    Redivo-Zaglia, M., Rodriguez, G.: smt: a matlab toolbox for structured matrices. Numer. Algorithms 59, 639–659 (2012)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Stoll, M.: A Krylov–Schur approach to the truncated SVD. Linear Algebra Appl. 436, 2795–2806 (2012)MathSciNetCrossRefMATHGoogle Scholar
  32. 32.
    Tang, J., Saad, Y.: A probing method for computing the diagonal of the matrix inverse. Numer. Linear Algebra Appl. 19, 485–501 (2012)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    van der Mee, C.V.M., Seatzu, S.: A method for generating infinite positive self-adjoint test matrices and Riesz bases. SIAM J. Matrix Anal. Appl. 26, 1132–1149 (2005)MathSciNetCrossRefMATHGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  1. 1.Dipartimento di InformaticaUniversità di PisaPisaItaly
  2. 2.Department of Mathematical SciencesKent State UniversityKentUSA
  3. 3.Dipartimento di Matematica e InformaticaUniversità di CagliariCagliariItaly
  4. 4.Laboratoire de Mathématiques Pures et AppliquéesUniversité du Littoral, Centre Universtaire de la Mi-VoixCalais cedexFrance

Personalised recommendations