Skip to main content

The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

Abstract

Regularization techniques based on the Golub-Kahan iterative bidiagonalization belong among popular approaches for solving large ill-posed problems. First, the original problem is projected onto a lower dimensional subspace using the bidiagonalization algorithm, which by itself represents a form of regularization by projection. The projected problem, however, inherits a part of the ill-posedness of the original problem, and therefore some form of inner regularization must be applied. Stopping criteria for the whole process are then based on the regularization of the projected (small) problem.

In this paper we consider an ill-posed problem with a noisy right-hand side (observation vector), where the noise level is unknown. We show how the information from the Golub-Kahan iterative bidiagonalization can be used for estimating the noise level. Such information can be useful for constructing efficient stopping criteria in solving ill-posed problems.

This is a preview of subscription content, access via your institution.

References

  1. 1.

    Björck, Å.: A bidiagonalization algorithm for solving large sparse ill-posed systems of linear equations. BIT 28, 659–670 (1988)

    MATH  Article  MathSciNet  Google Scholar 

  2. 2.

    Björck, Å., Grimme, E., Van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34, 510–534 (1994)

    MATH  Article  MathSciNet  Google Scholar 

  3. 3.

    Barlow, J.L., Bosner, N., Drmač, Z.: A new stable bidiagonal reduction algorithm. Linear Algebra Appl. 397, 35–84 (2005)

    MATH  Article  MathSciNet  Google Scholar 

  4. 4.

    Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999)

    MATH  Article  MathSciNet  Google Scholar 

  5. 5.

    Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000)

    MATH  Article  MathSciNet  Google Scholar 

  6. 6.

    Calvetti, D., Reichel, L., Shuibi, A.: L-curve and curvature bounds for Tikhonov regularization. Numer. Algorithms 35, 301–314 (2004)

    MATH  Article  MathSciNet  Google Scholar 

  7. 7.

    Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Electron. Trans. Numer. Anal. 28, 149–167 (2008)

    MathSciNet  Google Scholar 

  8. 8.

    Cooley, J.W., Tukey, J.W.: An algorithm for the machine computation of the complex Fourier series. Math. Comput. 19, 297–301 (1965)

    MATH  Article  MathSciNet  Google Scholar 

  9. 9.

    Duhamel, P., Vetterli, M.: Fast Fourier transforms: A tutorial review and a state of the art. Signal Process. 19, 259–299 (1990)

    MATH  Article  MathSciNet  Google Scholar 

  10. 10.

    Fierro, R.D., Golub, G.H., Hansen, P.C., O’Leary, D.P.: Regularization by truncated total least squares. SIAM J. Sci. Statist. Comput. 18, 1225–1241 (1997)

    Article  MathSciNet  Google Scholar 

  11. 11.

    Fischer, B.: Polynomial Based Iteration Methods for Symmetric Linear Systems. Wiley-Teubner Series Advances in Numerical Mathematics. Wiley, New York (1996)

    MATH  Google Scholar 

  12. 12.

    Fischer, B., Freund, R.W.: On adaptive weighted polynomial preconditioning for Hermitian positive definite matrices. SIAM J. Sci. Comput. 15, 408–426 (1994)

    MATH  Article  MathSciNet  Google Scholar 

  13. 13.

    Gautschi, W.: Gauss-Christoffel Quadrature Formulae. In: E.B. Christoffel, Aachen/Monschau, 1979, pp. 72–147. Birkhäuser, Basel (1981)

    Google Scholar 

  14. 14.

    Golub, G.H., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. SIAM J. Numer. Anal. Ser. B 2, 205–224 (1965)

    MathSciNet  Article  Google Scholar 

  15. 15.

    Golub, G.H., Von Matt, U.: Generalized cross-validation for large scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)

    Article  Google Scholar 

  16. 16.

    Hanke, M.: On Lanczos based methods for regularization of discrete ill-posed problems. BIT 41, 1008–1018 (2001)

    Article  MathSciNet  Google Scholar 

  17. 17.

    Hansen, P.C.: Regularization Tools—version 3.2 for MATLAB 6.0, a package for analysis and solution of discrete ill-posed problems, http://www2.imm.dtu.dk/~pch/Regutools/index.html

  18. 18.

    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems, Numerical Aspects of Linear Inversion. SIAM, Philadelphia (1998)

    Google Scholar 

  19. 19.

    Hansen, P.C., Jensen, T.K.: Noise propagation in regularizing iterations for image deblurring. ETNA 31, 204–220 (2008)

    MATH  MathSciNet  Google Scholar 

  20. 20.

    Hansen, P.C., Kilmer, M.E., Kjeldsen, R.: Exploiting residual information in the parameter choice for discrete ill-posed problems. BIT 46, 41–59 (2006)

    MATH  Article  MathSciNet  Google Scholar 

  21. 21.

    Hansen, P.C., Sørensen, H.O., Sükösd, Z., Poulsen, H.F.: Reconstruction of single-grain orientation distribution functions for crystalline materials. SIAM J. Image Sci. 2(2), 593–613 (2009)

    MATH  Article  Google Scholar 

  22. 22.

    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–435 (1952)

    MATH  MathSciNet  Google Scholar 

  23. 23.

    Hnětynková, I., Strakoš, Z.: Lanczos tridiagonalization and core problems. Linear Algebra Appl. 421, 243–251 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  24. 24.

    Hnětynková, I., Plešinger, M., Strakoš, Z.: Lanczos tridiagonalization, Golub-Kahan bidiagonalization and core problem. PAMM 6, 717–718 (2006)

    Article  Google Scholar 

  25. 25.

    Jensen, T.K., Hansen, P.C.: Iterative regularization with minimum-residual methods. BIT 47, 103–120 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  26. 26.

    Karlin, S., Shapley, L.S.: Geometry of Moment Spaces. American Mathematical Society, Providence (1953)

    Google Scholar 

  27. 27.

    Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 22, 1204–1221 (2001)

    MATH  Article  MathSciNet  Google Scholar 

  28. 28.

    Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general form Tikhonov regularization. SIAM J. Sci. Comput. 29, 315–330 (2006)

    Article  Google Scholar 

  29. 29.

    Lanczos, C.: Linear Differential Operators. Van Nostrand, London (1961)

    MATH  Google Scholar 

  30. 30.

    Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J. Res. Nat. Bur. Stand. 45, 255–282 (1950)

    MathSciNet  Google Scholar 

  31. 31.

    Meurant, G., Strakoš, Z.: The Lanczos and conjugate gradient algorithms in finite precision arithmetic. Acta Numer. 15, 471–542 (2006)

    MATH  Article  MathSciNet  Google Scholar 

  32. 32.

    Morozov, V.A.: On the solution of functional equations by the method of regularization (in Russian). Sov. Math. Dokl. 7, 414–417 (1966)

    MATH  Google Scholar 

  33. 33.

    Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984)

    Google Scholar 

  34. 34.

    Nguyen, N., Milanfar, P., Golub, G.H.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Proces. 10, 1299–1308 (2001)

    MATH  Article  MathSciNet  Google Scholar 

  35. 35.

    O’Leary, D.P.: Near-optimal parameters for Tikhonov and other regularization methods. SIAM J. Sci. Comput. 23, 1161–1171 (2001)

    MATH  Article  MathSciNet  Google Scholar 

  36. 36.

    O’Leary, D.P., Simmons, J.A.: A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2, 474–489 (1981)

    MATH  Article  MathSciNet  Google Scholar 

  37. 37.

    O’Leary, D.P., Strakoš, Z., Tichý, P.: On sensitivity of Gauss-Christoffel quadrature. Numer. Math. 107, 147–174 (2007)

    MATH  Article  MathSciNet  Google Scholar 

  38. 38.

    Paige, C.C.: Bidiagonalization of matrices and solution of linear equations. SIAM J. Numer. Anal. 11, 197–209 (1974)

    MATH  Article  MathSciNet  Google Scholar 

  39. 39.

    Paige, C.C.: Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem. Linear Algebra Appl. 34, 235–258 (1980)

    MATH  Article  MathSciNet  Google Scholar 

  40. 40.

    Paige, C.C.: A useful form of unitary matrix obtained from any sequence of unit 2-norm n-vectors. SIAM J. Matrix Anal. Appl. 31, 565–583 (2009)

    Article  MathSciNet  Google Scholar 

  41. 41.

    Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8, 43–71 (1982)

    MATH  Article  MathSciNet  Google Scholar 

  42. 42.

    Paige, C.C., Saunders, M.A.: ALGORITHM 583 LSQR: Sparse Linear equations and least squares problems. ACM Trans. Math. Softw. 8, 195–209 (1982)

    Article  MathSciNet  Google Scholar 

  43. 43.

    Paige, C.C., Strakoš, Z.: Scaled total least squares fundamentals. Numer. Math. 91, 117–146 (2002)

    MATH  Article  MathSciNet  Google Scholar 

  44. 44.

    Paige, C.C., Strakoš, Z.: Unifying least squares, total least squares and data least squares. In: Van Huffel, S., Lemmerling, P. (eds.) Total Least Squares and Errors-in-Variables Modeling, pp. 25–34. Kluwer Academic, Dordrecht (2002)

    Google Scholar 

  45. 45.

    Paige, C.C., Strakoš, Z.: Core problem in linear algebraic systems. SIAM J. Matrix Anal. Appl. 27, 861–875 (2006)

    MATH  Article  Google Scholar 

  46. 46.

    Rust, B.W.: Parameter selection for constrained solutions to ill-posed problems. Comput. Sci. Stat. 32, 333–347 (2000)

    Google Scholar 

  47. 47.

    Rust, B.W., O’Leary, D.P.: Residual periodograms for choosing regularization parameters for ill-posed problems. Inverse Probl. 24 (2008). doi:10.1088/0266-5611/24/3/034005

  48. 48.

    Saunders, M.A.: Computing projections with LSQR. BIT 37, 96–104 (1997)

    MATH  Article  MathSciNet  Google Scholar 

  49. 49.

    Sima, D.M., Van Huffel, S.: Using core formulations for ill-posed linear systems. PAMM 5, 795–796 (2005)

    Article  Google Scholar 

  50. 50.

    Sima, D.M.: Regularization techniques in model fitting and parameter estimation. Ph.D. thesis, Dept. of Electrical Engineering, Katholieke Universiteit Leuven (2006)

  51. 51.

    Simon, H.D., Zha, H.: Low-rank matrix approximation using the Lanczos bidiagonalization process with applications. SIAM J. Sci. Stat. Comput. 21, 2257–2274 (2000)

    MATH  Article  MathSciNet  Google Scholar 

  52. 52.

    Strakoš, Z.: Model reduction using the Vorobyev moment problem. Numer. Algorithms 51, 363–376 (2009)

    MATH  Article  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Iveta Hnětynková.

Additional information

The work of the first author was supported by the research project MSM0021620839 financed by MŠMT. The work of the second and the third author was supported by the GAAS grant IAA100300802, and by the Institutional Research Plan AV0Z10300504.

Communicated by Lars Eldén.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hnětynková, I., Plešinger, M. & Strakoš, Z. The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data. Bit Numer Math 49, 669–696 (2009). https://doi.org/10.1007/s10543-009-0239-7

Download citation

Keywords

  • Ill-posed problems
  • Golub-Kahan iterative bidiagonalization
  • Lanczos tridiagonalization
  • Noise revealing

Mathematics Subject Classification (2000)

  • 15A06
  • 15A18
  • 15A23
  • 65F10
  • 65F22