Skip to main content
SpringerLink
Log in

Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

For the accurate approximation of the minimal singular triple (singular value and left and right singular vector) of a large sparse matrix, we may use two separate search spaces, one for the left, and one for the right singular vector. In Lanczos bidiagonalization, for example, such search spaces are constructed. In SIAM J. Sci. Comput., 23(2) (2002), pp. 606–628, the author proposes a Jacobi–Davidson type method for the singular value problem, where solutions to certain correction equations are used to expand the search spaces.

As noted in the mentioned paper, the standard Galerkin subspace extraction works well for the computation of large singular triples, but may lead to unsatisfactory approximations to small and interior triples. To overcome this problem for the smallest triples, we propose three harmonic and a refined approach. All methods are derived in a number of different ways. Some of these methods can also be applied when we are interested in the largest or interior singular triples. Theoretical results as well as numerical experiments indicate that the results of the alternative extraction processes are often better than the standard approach. We show that when Lanczos bidiagonalization is used for the subspace expansion, the standard, harmonic, and refined extraction methods are all essentially equivalent. This gives more insight in the success of Lanczos bidiagonalization to find the smallest singular triples.

Finally, we show that the extraction processes for the smallest singular values may give an approximation to a least squares problem at low additional costs. The truncated SVD is also discussed in this context.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. Baglama, D. Calvetti, and L. Reichel, IRBL: An implicitly restarted Block–Lanczos method for large-scale Hermitian eigenproblems, SIAM J. Sci. Comput., 24(5) (2003), pp. 1650–1677.

    Google Scholar 

  2. J. Baglama and L. Reichel, Augmented implicitly restarted Lanczos bidiagonalization methods, Preprint, 2004.

  3. Z. Bai, J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst (eds.), Templates for the Solution of Algebraic Eigenvalue Problems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2000. A practical guide.

    Google Scholar 

  4. Å. Björck, Numerical Methods for Least Squares Problems, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1996.

    Google Scholar 

  5. G. H. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, J. Soc. Indust. Appl. Math. Ser. B Numer. Anal., 2 (1965), pp. 205–224.

    Google Scholar 

  6. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd edn, The John Hopkins University Press, Baltimore, 1996.

    Google Scholar 

  7. M. E. Hochstenbach, A Jacobi–Davidson type SVD method, SIAM J. Sci. Comput., 23(2) (2001), pp. 606–628.

    Google Scholar 

  8. JDQR code, available via http://www.math.uu.nl/people/sleijpen.

  9. Z. Jia, Refined iterative algorithms based on Arnoldi’s process for large unsymmetric eigenproblems, Linear Algebra Appl., 259 (1997), pp. 1–23.

    Google Scholar 

  10. Z. Jia and D. Niu, An implicitly restarted refined bidiagonalization Lanczos method for computing a partial singular value decomposition, SIAM J. Matrix Anal. Appl., 25(1) (2003), pp. 246–265.

    Google Scholar 

  11. A. V. Knyazev, New estimates for Ritz vectors, Math. Comp., 66(219) (1997), pp. 985–995.

    Google Scholar 

  12. E. Kokiopoulou, C. Bekas and E. Gallopoulos, Computing smallest singular triplets with implicitly restarted Lanczos bidiagonalization, Appl. Numer. Math., 49(1) (2004), pp. 39–61.

    Google Scholar 

  13. R. M. Larsen, Lanczos bidiagonalization with partial reorthogonalization, Technical report DAIMI PB-357, Department of Computer Science, University of Aarhus, September 1998. See also http://sun.stanford.edu/~rmunk/PROPACK.

  14. The Matrix Market, http://math.nist.gov/MatrixMarket, a repository for test matrices.

  15. R. B. Morgan, Computing interior eigenvalues of large matrices, Linear Algebra Appl., 154/156 (1991), pp. 289–309.

    Google Scholar 

  16. C. C. Paige, B. N. Parlett, and H. A. van der Vorst, Approximate solutions and eigenvalue bounds from Krylov subspaces, Num. Lin. Alg. Appl., 2(2) (1995), pp. 115–133.

    Google Scholar 

  17. C. C. Paige and M. A. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8(1) (1982), pp. 43–71.

    Google Scholar 

  18. B. N. Parlett, The Symmetric Eigenvalue Problem, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1998. Corrected reprint of the 1980 original.

    Google Scholar 

  19. Y. Saad, Numerical Methods for Large Eigenvalue Problems, Manchester University Press, Manchester, UK, 1992.

    Google Scholar 

  20. H. D. Simon and H. Zha, Low-rank matrix approximation using the Lanczos bidiagonalization process with applications, SIAM J. Sci. Comput., 21(6) (2000), pp. 2257–2274.

    Google Scholar 

  21. G. L. G. Sleijpen, H. A. van der Vorst, and E. Meijerink, Efficient expansion of subspaces in the Jacobi–Davidson method for standard and generalized eigenproblems, Electron. Trans. Numer. Anal., 7 (1998), pp. 75–89.

    Google Scholar 

  22. G. W. Stewart, Matrix Algorithms, Vol. II, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2001.

    Google Scholar 

  23. G. W. Stewart and J. G. Sun, Matrix Perturbation Theory, Academic Press Inc., Boston, MA, 1990.

    Google Scholar 

  24. L. N. Trefethen, Computation of pseudospectra, in Acta Numerica, Cambridge Univ. Press, Cambridge, 1999, pp. 247–295.

    Google Scholar 

  25. S. Van Huffel and J. Vandewalle, The Total Least Squares Problem, Frontiers in Applied Mathematics, Vol. 9, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1991.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Case Western Reserve University, 10900 Euclid Avenue, Cleveland, OH, 44106-7058, USA

    M. E. Hochstenbach

Authors
  1. M. E. Hochstenbach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. E. Hochstenbach.

Additional information

AMS subject classification (2000)

65F15, 65F50, (65F35, 93E24).

Submitted December 2002. Accepted October 2004. Communicated by Haesun Park.

M. E. Hochstenbach: The research of this author was supported in part by NSF grant DMS-0405387. Part of this work was done when the author was at Utrecht University.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hochstenbach, M.E. Harmonic and Refined Extraction Methods for the Singular Value Problem, with Applications in Least Squares Problems. Bit Numer Math 44, 721–754 (2004). https://doi.org/10.1007/s10543-004-5244-2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-004-5244-2

Keywords

Search

Navigation