Skip to main content

The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

BIT Numerical Mathematics volume 49, pages 669–696 (2009)Cite this article

Abstract

Regularization techniques based on the Golub-Kahan iterative bidiagonalization belong among popular approaches for solving large ill-posed problems. First, the original problem is projected onto a lower dimensional subspace using the bidiagonalization algorithm, which by itself represents a form of regularization by projection. The projected problem, however, inherits a part of the ill-posedness of the original problem, and therefore some form of inner regularization must be applied. Stopping criteria for the whole process are then based on the regularization of the projected (small) problem.

In this paper we consider an ill-posed problem with a noisy right-hand side (observation vector), where the noise level is unknown. We show how the information from the Golub-Kahan iterative bidiagonalization can be used for estimating the noise level. Such information can be useful for constructing efficient stopping criteria in solving ill-posed problems.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Björck, Å.: A bidiagonalization algorithm for solving large sparse ill-posed systems of linear equations. BIT 28, 659–670 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  2. Björck, Å., Grimme, E., Van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34, 510–534 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  3. Barlow, J.L., Bosner, N., Drmač, Z.: A new stable bidiagonal reduction algorithm. Linear Algebra Appl. 397, 35–84 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  4. Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  5. Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  6. Calvetti, D., Reichel, L., Shuibi, A.: L-curve and curvature bounds for Tikhonov regularization. Numer. Algorithms 35, 301–314 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  7. Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Electron. Trans. Numer. Anal. 28, 149–167 (2008)

    MathSciNet  Google Scholar 

  8. Cooley, J.W., Tukey, J.W.: An algorithm for the machine computation of the complex Fourier series. Math. Comput. 19, 297–301 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  9. Duhamel, P., Vetterli, M.: Fast Fourier transforms: A tutorial review and a state of the art. Signal Process. 19, 259–299 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fierro, R.D., Golub, G.H., Hansen, P.C., O’Leary, D.P.: Regularization by truncated total least squares. SIAM J. Sci. Statist. Comput. 18, 1225–1241 (1997)

    Article  MathSciNet  Google Scholar 

  11. Fischer, B.: Polynomial Based Iteration Methods for Symmetric Linear Systems. Wiley-Teubner Series Advances in Numerical Mathematics. Wiley, New York (1996)

    MATH  Google Scholar 

  12. Fischer, B., Freund, R.W.: On adaptive weighted polynomial preconditioning for Hermitian positive definite matrices. SIAM J. Sci. Comput. 15, 408–426 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  13. Gautschi, W.: Gauss-Christoffel Quadrature Formulae. In: E.B. Christoffel, Aachen/Monschau, 1979, pp. 72–147. Birkhäuser, Basel (1981)

    Google Scholar 

  14. Golub, G.H., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. SIAM J. Numer. Anal. Ser. B 2, 205–224 (1965)

    Article  MathSciNet  Google Scholar 

  15. Golub, G.H., Von Matt, U.: Generalized cross-validation for large scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)

    Article  Google Scholar 

  16. Hanke, M.: On Lanczos based methods for regularization of discrete ill-posed problems. BIT 41, 1008–1018 (2001)

    Article  MathSciNet  Google Scholar 

  17. Hansen, P.C.: Regularization Tools—version 3.2 for MATLAB 6.0, a package for analysis and solution of discrete ill-posed problems, http://www2.imm.dtu.dk/~pch/Regutools/index.html

  18. Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems, Numerical Aspects of Linear Inversion. SIAM, Philadelphia (1998)

    Google Scholar 

  19. Hansen, P.C., Jensen, T.K.: Noise propagation in regularizing iterations for image deblurring. ETNA 31, 204–220 (2008)

    MATH  MathSciNet  Google Scholar 

  20. Hansen, P.C., Kilmer, M.E., Kjeldsen, R.: Exploiting residual information in the parameter choice for discrete ill-posed problems. BIT 46, 41–59 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  21. Hansen, P.C., Sørensen, H.O., Sükösd, Z., Poulsen, H.F.: Reconstruction of single-grain orientation distribution functions for crystalline materials. SIAM J. Image Sci. 2(2), 593–613 (2009)

    Article  MATH  Google Scholar 

  22. Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–435 (1952)

    MATH  MathSciNet  Google Scholar 

  23. Hnětynková, I., Strakoš, Z.: Lanczos tridiagonalization and core problems. Linear Algebra Appl. 421, 243–251 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  24. Hnětynková, I., Plešinger, M., Strakoš, Z.: Lanczos tridiagonalization, Golub-Kahan bidiagonalization and core problem. PAMM 6, 717–718 (2006)

    Article  Google Scholar 

  25. Jensen, T.K., Hansen, P.C.: Iterative regularization with minimum-residual methods. BIT 47, 103–120 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  26. Karlin, S., Shapley, L.S.: Geometry of Moment Spaces. American Mathematical Society, Providence (1953)

    Google Scholar 

  27. Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 22, 1204–1221 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  28. Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general form Tikhonov regularization. SIAM J. Sci. Comput. 29, 315–330 (2006)

    Article  Google Scholar 

  29. Lanczos, C.: Linear Differential Operators. Van Nostrand, London (1961)

    MATH  Google Scholar 

  30. Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J. Res. Nat. Bur. Stand. 45, 255–282 (1950)

    MathSciNet  Google Scholar 

  31. Meurant, G., Strakoš, Z.: The Lanczos and conjugate gradient algorithms in finite precision arithmetic. Acta Numer. 15, 471–542 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  32. Morozov, V.A.: On the solution of functional equations by the method of regularization (in Russian). Sov. Math. Dokl. 7, 414–417 (1966)

    MATH  Google Scholar 

  33. Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984)

    Google Scholar 

  34. Nguyen, N., Milanfar, P., Golub, G.H.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Proces. 10, 1299–1308 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  35. O’Leary, D.P.: Near-optimal parameters for Tikhonov and other regularization methods. SIAM J. Sci. Comput. 23, 1161–1171 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  36. O’Leary, D.P., Simmons, J.A.: A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2, 474–489 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  37. O’Leary, D.P., Strakoš, Z., Tichý, P.: On sensitivity of Gauss-Christoffel quadrature. Numer. Math. 107, 147–174 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  38. Paige, C.C.: Bidiagonalization of matrices and solution of linear equations. SIAM J. Numer. Anal. 11, 197–209 (1974)

    Article  MATH  MathSciNet  Google Scholar 

  39. Paige, C.C.: Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem. Linear Algebra Appl. 34, 235–258 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  40. Paige, C.C.: A useful form of unitary matrix obtained from any sequence of unit 2-norm n-vectors. SIAM J. Matrix Anal. Appl. 31, 565–583 (2009)

    Article  MathSciNet  Google Scholar 

  41. Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8, 43–71 (1982)

    Article  MATH  MathSciNet  Google Scholar 

  42. Paige, C.C., Saunders, M.A.: ALGORITHM 583 LSQR: Sparse Linear equations and least squares problems. ACM Trans. Math. Softw. 8, 195–209 (1982)

    Article  MathSciNet  Google Scholar 

  43. Paige, C.C., Strakoš, Z.: Scaled total least squares fundamentals. Numer. Math. 91, 117–146 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  44. Paige, C.C., Strakoš, Z.: Unifying least squares, total least squares and data least squares. In: Van Huffel, S., Lemmerling, P. (eds.) Total Least Squares and Errors-in-Variables Modeling, pp. 25–34. Kluwer Academic, Dordrecht (2002)

    Google Scholar 

  45. Paige, C.C., Strakoš, Z.: Core problem in linear algebraic systems. SIAM J. Matrix Anal. Appl. 27, 861–875 (2006)

    Article  MATH  Google Scholar 

  46. Rust, B.W.: Parameter selection for constrained solutions to ill-posed problems. Comput. Sci. Stat. 32, 333–347 (2000)

    Google Scholar 

  47. Rust, B.W., O’Leary, D.P.: Residual periodograms for choosing regularization parameters for ill-posed problems. Inverse Probl. 24 (2008). doi:10.1088/0266-5611/24/3/034005

  48. Saunders, M.A.: Computing projections with LSQR. BIT 37, 96–104 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  49. Sima, D.M., Van Huffel, S.: Using core formulations for ill-posed linear systems. PAMM 5, 795–796 (2005)

    Article  Google Scholar 

  50. Sima, D.M.: Regularization techniques in model fitting and parameter estimation. Ph.D. thesis, Dept. of Electrical Engineering, Katholieke Universiteit Leuven (2006)

  51. Simon, H.D., Zha, H.: Low-rank matrix approximation using the Lanczos bidiagonalization process with applications. SIAM J. Sci. Stat. Comput. 21, 2257–2274 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  52. Strakoš, Z.: Model reduction using the Vorobyev moment problem. Numer. Algorithms 51, 363–376 (2009)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, Academy of Sciences, Pod Vodárenskou věží 2, Prague 8, Czech Republic

    Iveta Hnětynková, Martin Plešinger & Zdeněk Strakoš

  2. Faculty of Mathematics and Physics, Charles University in Prague, Sokolovská 83, Prague 8, Czech Republic

    Iveta Hnětynková & Zdeněk Strakoš

  3. Faculty of Mechatronics, Technical University of Liberec, Studentská 2/1402, Liberec, Czech Republic

    Martin Plešinger

Authors
  1. Iveta Hnětynková
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin Plešinger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zdeněk Strakoš
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Iveta Hnětynková.

Additional information

Communicated by Lars Eldén.

The work of the first author was supported by the research project MSM0021620839 financed by MŠMT. The work of the second and the third author was supported by the GAAS grant IAA100300802, and by the Institutional Research Plan AV0Z10300504.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Hnětynková, I., Plešinger, M. & Strakoš, Z. The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data. Bit Numer Math 49, 669–696 (2009). https://doi.org/10.1007/s10543-009-0239-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-009-0239-7

Keywords

  • Ill-posed problems
  • Golub-Kahan iterative bidiagonalization
  • Lanczos tridiagonalization
  • Noise revealing

Mathematics Subject Classification (2000)

  • 15A06
  • 15A18
  • 15A23
  • 65F10
  • 65F22