Skip to main content
Log in

A contribution to the theory and practice of the block Kogbetliantz method for computing the SVD

  • Published:
BIT Numerical Mathematics Aims and scope Submit manuscript

Abstract

This article studies the convergence and practical implementation of the block version of the Kogbetliantz algorithm for computing the singular value decomposition (SVD) of general real or complex matrices. Global convergence is proved for simple singular values and the singular vectors, including the asymptotically quadratic reduction to diagonal form. The convergence can be guaranteed for certain parallel block pivot strategies as well. This bridges a theoretical gap, provides solid theoretical basis for parallel implementations of the block algorithm, and provides valuable insights.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2

Similar content being viewed by others

Notes

  1. Software implementation of the Businger–Golub pivoting is numerically reliable with a modification from [13], which is included in LAPACK starting with the release 3.1.0.

  2. We do so because the absence of the triangular structure causes no additional technical difficulties.

  3. Paige attributed the idea to M. Gentleman.

References

  1. Anderson, E., Bai, Z., Bischof, C., Demmel, J., Dongarra, J., Croz, J.D., Greenbaum, A., Hammarling, S., McKenny, A., Ostrouchov, S., Sorensen, D.: LAPACK Users’ Guide, 2nd edn. SIAM, Philadelphia (1992)

    MATH  Google Scholar 

  2. Bai, Z.: Note on the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Linear Algebra Appl. 104, 131–140 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  3. Barlow, J.: More accurate bidiagonal reduction for computing the singular value decomposition. SIAM J. Matrix Anal. Appl. 23, 761–798 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  4. Barlow, J., Demmel, J.: Computing accurate eigensystems of scaled diagonally dominant matrices. SIAM J. Numer. Anal. 27(3), 762–791 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  5. Bečka, M., Okša, G., Vajteršic, M.: Dynamic ordering for a parallel block-Jacobi SVD algorithm. Parallel Comput. 28, 243–262 (2002)

    Article  MATH  Google Scholar 

  6. Brent, R.P., Luk, F.T., Van Loan, C.F.: Computation of the singular value decomposition using mesh-connected processors. J. VLSI Comput. Syst. 1(3), 242–270 (1985)

    MathSciNet  MATH  Google Scholar 

  7. Businger, P.A., Golub, G.H.: Linear least squares solutions by Householder transformations. Numer. Math. 7, 269–276 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  8. Causey, R.L., Henrici, P.: Convergence of approximate eigenvectors in Jacobi methods. Numer. Math. 2, 67–78 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chan, T.F.: An improved algorithm for computing the singular value decomposition. ACM Trans. Math. Softw. 8, 72–83 (1982)

    Article  MATH  Google Scholar 

  10. Charlier, J.P., Vanbegin, M., Dooren, P.V.: On efficient implementations of Kogbetliantz’s algorithm for computing the singular value decomposition. Numer. Math. 52, 279–300 (1988)

    Article  MATH  Google Scholar 

  11. Demmel, J., Veselić, K.: Jacobi’s method is more accurate than QR. SIAM J. Matrix Anal. Appl. 13(4), 1204–1245 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  12. Drmač, Z.: A global convergence proof for cyclic Jacobi methods with block rotations. SIAM J. Matrix Anal. Appl. 31, 1329–1350 (2011)

    Article  Google Scholar 

  13. Drmač, Z., Bujanović, Z.: On the failure of rank revealing QR factorization software—a case study. ACM Trans. Math. Softw. 35(2), 1–28 (2008)

    Article  Google Scholar 

  14. Drmač, Z., Veselić, K.: New fast and accurate Jacobi SVD algorithm: I. SIAM J. Matrix Anal. Appl. 29(4), 1322–1342 (2008)

    Article  MATH  Google Scholar 

  15. Drmač, Z., Veselić, K.: New fast and accurate Jacobi SVD algorithm: II. SIAM J. Matrix Anal. Appl. 29(4), 1343–1362 (2008)

    Article  MATH  Google Scholar 

  16. Fernando, K.V.: Linear convergence of the row-cyclic Jacobi and Kogbetliantz methods. Numer. Math. 56, 71–91 (1989)

    Article  MathSciNet  Google Scholar 

  17. Fernando, K.V., Hammarling, S.J.: Kogbetliantz methods for parallel SVD computation: architecture, algorithms and convergence. Tech. Rep. TR9/86, NAG Limited Oxford (1986). (FLAME Working Note 9.)

  18. Forsythe, G.E., Henrici, P.: The cyclic Jacobi method for computing the principal values of a complex matrix. Trans. Am. Math. Soc. 94(1), 1–23 (1960)

    Article  MathSciNet  MATH  Google Scholar 

  19. Hari, V.: On the quadratic convergence bounds of the serial singular value decomposition by Jacobi methods for triangular matrices. SIAM J. Sci. Stat. Comput. 10, 1076–1096 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hari, V., Veselić, K.: On Jacobi methods for singular value decompositions. SIAM J. Sci. Stat. Comput. 8(5), 741–754 (1987)

    Article  MATH  Google Scholar 

  21. Hari, V., Zadelj-Martić, V.: Parallelizing the Kogbetliantz method: a first attempt. J. Numer. Anal. Ind. Appl. Math. 2(1), 49–66 (2007)

    MathSciNet  MATH  Google Scholar 

  22. Kogbetliantz, E.G.: Solution of linear equations by diagonalization of coefficient matrix. Q. Appl. Math. 13, 123–132 (1955)

    MathSciNet  MATH  Google Scholar 

  23. Luk, F.: A triangular processor array for computing singular values. Linear Algebra Appl. 77, 259–273 (1986)

    Article  MATH  Google Scholar 

  24. Luk, F., Park, H.: On parallel Jacobi orderings. SIAM J. Sci. Stat. Comput. 10, 18–26 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  25. Matejaš, J., Hari, V.: Accuracy of the Kogbetliantz method for scaled diagonally dominant triangular matrices. Appl. Comput. Math. 217(8), 3726–3746 (2010)

    Article  MATH  Google Scholar 

  26. von Neumann, J.: On infinite direct products. Compos. Math. 6, 1–77 (1939)

    Google Scholar 

  27. Okša, G., Vajteršic, M.: Special issue: A systolic block-Jacobi SVD solver for processor meshes. Parallel Algorithms Appl. 18(1–2), 49–70 (2003)

    MathSciNet  MATH  Google Scholar 

  28. Okša, G., Vajteršic, M.: Efficient preprocessing in the parallel block-Jacobi SVD algorithm. Parallel Comput. 31, 166–176 (2005)

    Google Scholar 

  29. Paige, C.C.: Computing the generalized singular value decomposition. SIAM J. Sci. Stat. Comput. 7, 1126–1146 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  30. Paige, C.C., Dooren, P.V.: On the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Linear Algebra Appl. 7, 301–313 (1986)

    Article  Google Scholar 

  31. Paige, C.C., Wei, M.: History and generality of the CS decomposition. Linear Algebra Appl. 208(209), 303–326 (1994)

    Article  MathSciNet  Google Scholar 

  32. Stewart, G.W.: Computing the CS decomposition of a partitioned orthonormal matrix. Numer. Math. 40, 297–306 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  33. Stewart, G.W.: A Jacobi-like algorithm for computing the Schur decomposition of a non-Hermitian matrix. SIAM J. Sci. Stat. Comput. 6, 853–864 (1985)

    Article  MATH  Google Scholar 

  34. Stewart, G.W.: A gap-revealing matrix decomposition. Technical report TR-3771, Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 (1997)

  35. Stewart, G.W.: The QLP approximation to the singular value decomposition. Technical report TR-97-75, Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 (1997)

  36. Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, San Diego (1990)

    MATH  Google Scholar 

Download references

Acknowledgement

This work is supported by the Croatian MZOS grant 0372783-2750. The authors would like to thank the anonymous referees for their insightful comments and constructive criticism, which helped in improving this paper.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zvonimir Bujanović.

Additional information

Communicated by Daniel Kressner.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bujanović, Z., Drmač, Z. A contribution to the theory and practice of the block Kogbetliantz method for computing the SVD. Bit Numer Math 52, 827–849 (2012). https://doi.org/10.1007/s10543-012-0388-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10543-012-0388-y

Keywords

Mathematics Subject Classification (2010)

Navigation