Abstract
Tikhonov regularization is commonly used for the solution of linear discrete ill-posed problems with error-contaminated data. A regularization parameter that determines the quality of the computed solution has to be chosen. One of the most popular approaches to choosing this parameter is to minimize the Generalized Cross Validation (GCV) function. The minimum can be determined quite inexpensively when the matrix A that defines the linear discrete ill-posed problem is small enough to rapidly compute its singular value decomposition (SVD). We are interested in the solution of linear discrete ill-posed problems with a matrix A that is too large to make the computation of its complete SVD feasible, and show how upper and lower bounds for the numerator and denominator of the GCV function can be determined fairly inexpensively for large matrices A by computing only a few of the largest singular values and associated singular vectors of A. These bounds are used to determine a suitable value of the regularization parameter. Computed examples illustrate the performance of the proposed method.
Similar content being viewed by others
References
Avron, H., Toledo, S.: Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix. J. ACM 58, 8:1–8:34 (2011)
Baglama, J., Fenu, C., Reichel, L., Rodriguez, G.: Analysis of directed networks via partial singular value decomposition and Gauss quadrature. Linear Algebra Appl. 456, 93–121 (2014)
Baglama, J., Reichel, L.: Restarted block Lanczos bidiagonalization methods. Numer. Algorithms 43, 251–272 (2006)
Baglama, J., Reichel, L.: An implicitly restarted block Lanczos bidiagonalization method using Leja shifts. BIT 53, 285–310 (2013)
Bai, Z., Fahey, M., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74, 71–89 (1996)
Björck, Å.: A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations. BIT 18, 659–670 (1988)
Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Brezinski, C., Fika, P., Mitrouli, M.: Estimations of the trace of powers of self-adjoint operators by extrapolation of the moments. Electron. Trans. Numer. Anal. 39, 144–159 (2012)
Brezinski, C., Fika, P., Mitrouli, M.: Moments of a linear operator on a Hilbert space, with applications to the trace of the inverse of matrices and the solution of equations. Numer. Linear Algebra Appl. 19, 937–953 (2012)
Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31, 377–403 (1979)
Eldén, L.: A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT 22, 487–501 (1982)
Eldén, L.: A note on the computation of the generalized cross-validation function for ill-conditioned least squares problems. BIT 24, 467–472 (1984)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Fenu, C., Reichel, L., Rodriguez, G.: GCV for Tikhonov regularization via global Golub–Kahan decomposition. Numer. Linear Algebra Appl. 23, 467–484 (2016)
Gazzola, S., Novati, P., Russo, M.R.: On Krylov projection methods and Tikhonov regularization. Electron. Trans. Numer. Anal. 44, 83–123 (2015)
Golub, G.H., Heath, M., Wahba, G.: Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 21, 215–223 (1979)
Golub, G.H., Luk, F.T., Overton, M.L.: A block Lanczos method for computing the singular values and corresponding singular Vectors of a matrix. ACM Trans. Math. Software 7, 149–169 (1981)
Golub, G.H., Meurant, G.: Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)
Golub, G.H., von Matt, U.: Generalized cross-validation for large-scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)
Hansen, P.C.: Regularization tools version 4.0 for MATLAB 7.3. Numer. Algorithms 46, 189–194 (2007)
Hochstenbach, M.E.: A Jacobi–Davidson type SVD method. SIAM J. Sci. Comput. 23, 606–628 (2001)
Hochstenbach, M.E.: Harmonic and refined extraction methods for the singular value problem, with applications in least squares problems. BIT 44, 721–754 (2004)
Hutchinson, M.F.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Statist. Simula. 18, 1059–1076 (1989)
Jia, Z., Niu, D.: An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition. SIAM J. Matrix Anal. Appl. 25, 246–265 (2003)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)
Morigi, S., Reichel, L., Sgallari, F.: Orthogonal projection regularization operators. Numer. Algorithms 44, 99–114 (2007)
Novati, P., Russo, M.R.: A GCV based Arnoldi–Tikhonov regularization method. BIT 54, 501–521 (2014)
Onunwor, E., Reichel, L.: On the computation of a truncated SVD of a large linear discrete ill-posed problem. Numer. Algorithms, in press
Redivo-Zaglia, M., Rodriguez, G.: smt: a matlab toolbox for structured matrices. Numer. Algorithms 59, 639–659 (2012)
Stoll, M.: A Krylov–Schur approach to the truncated SVD. Linear Algebra Appl. 436, 2795–2806 (2012)
Tang, J., Saad, Y.: A probing method for computing the diagonal of the matrix inverse. Numer. Linear Algebra Appl. 19, 485–501 (2012)
van der Mee, C.V.M., Seatzu, S.: A method for generating infinite positive self-adjoint test matrices and Riesz bases. SIAM J. Matrix Anal. Appl. 26, 1132–1149 (2005)
Acknowledgements
The authors would like to thank Michiel Hochstenbach and a referee for comments. Work of C.F and G.R. was partially supported by INdAM-GNCS.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Michiel Hochstenbach.
Rights and permissions
About this article
Cite this article
Fenu, C., Reichel, L., Rodriguez , G. et al. GCV for Tikhonov regularization by partial SVD. Bit Numer Math 57, 1019–1039 (2017). https://doi.org/10.1007/s10543-017-0662-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10543-017-0662-0