BIT Numerical Mathematics

, Volume 49, Issue 4, pp 669–696 | Cite as

The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data

  • Iveta HnětynkováEmail author
  • Martin Plešinger
  • Zdeněk Strakoš


Regularization techniques based on the Golub-Kahan iterative bidiagonalization belong among popular approaches for solving large ill-posed problems. First, the original problem is projected onto a lower dimensional subspace using the bidiagonalization algorithm, which by itself represents a form of regularization by projection. The projected problem, however, inherits a part of the ill-posedness of the original problem, and therefore some form of inner regularization must be applied. Stopping criteria for the whole process are then based on the regularization of the projected (small) problem.

In this paper we consider an ill-posed problem with a noisy right-hand side (observation vector), where the noise level is unknown. We show how the information from the Golub-Kahan iterative bidiagonalization can be used for estimating the noise level. Such information can be useful for constructing efficient stopping criteria in solving ill-posed problems.


Ill-posed problems Golub-Kahan iterative bidiagonalization Lanczos tridiagonalization Noise revealing 

Mathematics Subject Classification (2000)

15A06 15A18 15A23 65F10 65F22 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Björck, Å.: A bidiagonalization algorithm for solving large sparse ill-posed systems of linear equations. BIT 28, 659–670 (1988) zbMATHCrossRefMathSciNetGoogle Scholar
  2. 2.
    Björck, Å., Grimme, E., Van Dooren, P.: An implicit shift bidiagonalization algorithm for ill-posed systems. BIT 34, 510–534 (1994) zbMATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Barlow, J.L., Bosner, N., Drmač, Z.: A new stable bidiagonal reduction algorithm. Linear Algebra Appl. 397, 35–84 (2005) zbMATHCrossRefMathSciNetGoogle Scholar
  4. 4.
    Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999) zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Calvetti, D., Morigi, S., Reichel, L., Sgallari, F.: Tikhonov regularization and the L-curve for large discrete ill-posed problems. J. Comput. Appl. Math. 123, 423–446 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Calvetti, D., Reichel, L., Shuibi, A.: L-curve and curvature bounds for Tikhonov regularization. Numer. Algorithms 35, 301–314 (2004) zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Chung, J., Nagy, J.G., O’Leary, D.P.: A weighted GCV method for Lanczos hybrid regularization. Electron. Trans. Numer. Anal. 28, 149–167 (2008) MathSciNetGoogle Scholar
  8. 8.
    Cooley, J.W., Tukey, J.W.: An algorithm for the machine computation of the complex Fourier series. Math. Comput. 19, 297–301 (1965) zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Duhamel, P., Vetterli, M.: Fast Fourier transforms: A tutorial review and a state of the art. Signal Process. 19, 259–299 (1990) zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Fierro, R.D., Golub, G.H., Hansen, P.C., O’Leary, D.P.: Regularization by truncated total least squares. SIAM J. Sci. Statist. Comput. 18, 1225–1241 (1997) CrossRefMathSciNetGoogle Scholar
  11. 11.
    Fischer, B.: Polynomial Based Iteration Methods for Symmetric Linear Systems. Wiley-Teubner Series Advances in Numerical Mathematics. Wiley, New York (1996) zbMATHGoogle Scholar
  12. 12.
    Fischer, B., Freund, R.W.: On adaptive weighted polynomial preconditioning for Hermitian positive definite matrices. SIAM J. Sci. Comput. 15, 408–426 (1994) zbMATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Gautschi, W.: Gauss-Christoffel Quadrature Formulae. In: E.B. Christoffel, Aachen/Monschau, 1979, pp. 72–147. Birkhäuser, Basel (1981) Google Scholar
  14. 14.
    Golub, G.H., Kahan, W.: Calculating the singular values and pseudo-inverse of a matrix. SIAM J. Numer. Anal. Ser. B 2, 205–224 (1965) MathSciNetCrossRefGoogle Scholar
  15. 15.
    Golub, G.H., Von Matt, U.: Generalized cross-validation for large scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997) CrossRefGoogle Scholar
  16. 16.
    Hanke, M.: On Lanczos based methods for regularization of discrete ill-posed problems. BIT 41, 1008–1018 (2001) CrossRefMathSciNetGoogle Scholar
  17. 17.
    Hansen, P.C.: Regularization Tools—version 3.2 for MATLAB 6.0, a package for analysis and solution of discrete ill-posed problems,
  18. 18.
    Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems, Numerical Aspects of Linear Inversion. SIAM, Philadelphia (1998) Google Scholar
  19. 19.
    Hansen, P.C., Jensen, T.K.: Noise propagation in regularizing iterations for image deblurring. ETNA 31, 204–220 (2008) zbMATHMathSciNetGoogle Scholar
  20. 20.
    Hansen, P.C., Kilmer, M.E., Kjeldsen, R.: Exploiting residual information in the parameter choice for discrete ill-posed problems. BIT 46, 41–59 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Hansen, P.C., Sørensen, H.O., Sükösd, Z., Poulsen, H.F.: Reconstruction of single-grain orientation distribution functions for crystalline materials. SIAM J. Image Sci. 2(2), 593–613 (2009) zbMATHCrossRefGoogle Scholar
  22. 22.
    Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49, 409–435 (1952) zbMATHMathSciNetGoogle Scholar
  23. 23.
    Hnětynková, I., Strakoš, Z.: Lanczos tridiagonalization and core problems. Linear Algebra Appl. 421, 243–251 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  24. 24.
    Hnětynková, I., Plešinger, M., Strakoš, Z.: Lanczos tridiagonalization, Golub-Kahan bidiagonalization and core problem. PAMM 6, 717–718 (2006) CrossRefGoogle Scholar
  25. 25.
    Jensen, T.K., Hansen, P.C.: Iterative regularization with minimum-residual methods. BIT 47, 103–120 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  26. 26.
    Karlin, S., Shapley, L.S.: Geometry of Moment Spaces. American Mathematical Society, Providence (1953) Google Scholar
  27. 27.
    Kilmer, M.E., O’Leary, D.P.: Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 22, 1204–1221 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  28. 28.
    Kilmer, M.E., Hansen, P.C., Español, M.I.: A projection-based approach to general form Tikhonov regularization. SIAM J. Sci. Comput. 29, 315–330 (2006) CrossRefGoogle Scholar
  29. 29.
    Lanczos, C.: Linear Differential Operators. Van Nostrand, London (1961) zbMATHGoogle Scholar
  30. 30.
    Lanczos, C.: An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. J. Res. Nat. Bur. Stand. 45, 255–282 (1950) MathSciNetGoogle Scholar
  31. 31.
    Meurant, G., Strakoš, Z.: The Lanczos and conjugate gradient algorithms in finite precision arithmetic. Acta Numer. 15, 471–542 (2006) zbMATHCrossRefMathSciNetGoogle Scholar
  32. 32.
    Morozov, V.A.: On the solution of functional equations by the method of regularization (in Russian). Sov. Math. Dokl. 7, 414–417 (1966) zbMATHGoogle Scholar
  33. 33.
    Morozov, V.A.: Methods for Solving Incorrectly Posed Problems. Springer, New York (1984) Google Scholar
  34. 34.
    Nguyen, N., Milanfar, P., Golub, G.H.: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Trans. Image Proces. 10, 1299–1308 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  35. 35.
    O’Leary, D.P.: Near-optimal parameters for Tikhonov and other regularization methods. SIAM J. Sci. Comput. 23, 1161–1171 (2001) zbMATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    O’Leary, D.P., Simmons, J.A.: A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 2, 474–489 (1981) zbMATHCrossRefMathSciNetGoogle Scholar
  37. 37.
    O’Leary, D.P., Strakoš, Z., Tichý, P.: On sensitivity of Gauss-Christoffel quadrature. Numer. Math. 107, 147–174 (2007) zbMATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    Paige, C.C.: Bidiagonalization of matrices and solution of linear equations. SIAM J. Numer. Anal. 11, 197–209 (1974) zbMATHCrossRefMathSciNetGoogle Scholar
  39. 39.
    Paige, C.C.: Accuracy and effectiveness of the Lanczos algorithm for the symmetric eigenproblem. Linear Algebra Appl. 34, 235–258 (1980) zbMATHCrossRefMathSciNetGoogle Scholar
  40. 40.
    Paige, C.C.: A useful form of unitary matrix obtained from any sequence of unit 2-norm n-vectors. SIAM J. Matrix Anal. Appl. 31, 565–583 (2009) CrossRefMathSciNetGoogle Scholar
  41. 41.
    Paige, C.C., Saunders, M.A.: LSQR: An algorithm for sparse linear equations and sparse least squares. ACM Trans. Math. Softw. 8, 43–71 (1982) zbMATHCrossRefMathSciNetGoogle Scholar
  42. 42.
    Paige, C.C., Saunders, M.A.: ALGORITHM 583 LSQR: Sparse Linear equations and least squares problems. ACM Trans. Math. Softw. 8, 195–209 (1982) CrossRefMathSciNetGoogle Scholar
  43. 43.
    Paige, C.C., Strakoš, Z.: Scaled total least squares fundamentals. Numer. Math. 91, 117–146 (2002) zbMATHCrossRefMathSciNetGoogle Scholar
  44. 44.
    Paige, C.C., Strakoš, Z.: Unifying least squares, total least squares and data least squares. In: Van Huffel, S., Lemmerling, P. (eds.) Total Least Squares and Errors-in-Variables Modeling, pp. 25–34. Kluwer Academic, Dordrecht (2002) Google Scholar
  45. 45.
    Paige, C.C., Strakoš, Z.: Core problem in linear algebraic systems. SIAM J. Matrix Anal. Appl. 27, 861–875 (2006) zbMATHCrossRefGoogle Scholar
  46. 46.
    Rust, B.W.: Parameter selection for constrained solutions to ill-posed problems. Comput. Sci. Stat. 32, 333–347 (2000) Google Scholar
  47. 47.
    Rust, B.W., O’Leary, D.P.: Residual periodograms for choosing regularization parameters for ill-posed problems. Inverse Probl. 24 (2008). doi: 10.1088/0266-5611/24/3/034005
  48. 48.
    Saunders, M.A.: Computing projections with LSQR. BIT 37, 96–104 (1997) zbMATHCrossRefMathSciNetGoogle Scholar
  49. 49.
    Sima, D.M., Van Huffel, S.: Using core formulations for ill-posed linear systems. PAMM 5, 795–796 (2005) CrossRefGoogle Scholar
  50. 50.
    Sima, D.M.: Regularization techniques in model fitting and parameter estimation. Ph.D. thesis, Dept. of Electrical Engineering, Katholieke Universiteit Leuven (2006) Google Scholar
  51. 51.
    Simon, H.D., Zha, H.: Low-rank matrix approximation using the Lanczos bidiagonalization process with applications. SIAM J. Sci. Stat. Comput. 21, 2257–2274 (2000) zbMATHCrossRefMathSciNetGoogle Scholar
  52. 52.
    Strakoš, Z.: Model reduction using the Vorobyev moment problem. Numer. Algorithms 51, 363–376 (2009) zbMATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science + Business Media B.V. 2009

Authors and Affiliations

  • Iveta Hnětynková
    • 1
    • 2
    Email author
  • Martin Plešinger
    • 1
    • 3
  • Zdeněk Strakoš
    • 1
    • 2
  1. 1.Institute of Computer ScienceAcademy of SciencesPrague 8Czech Republic
  2. 2.Faculty of Mathematics and PhysicsCharles University in PraguePrague 8Czech Republic
  3. 3.Faculty of MechatronicsTechnical University of LiberecLiberecCzech Republic

Personalised recommendations