Abstract
This article studies the convergence and practical implementation of the block version of the Kogbetliantz algorithm for computing the singular value decomposition (SVD) of general real or complex matrices. Global convergence is proved for simple singular values and the singular vectors, including the asymptotically quadratic reduction to diagonal form. The convergence can be guaranteed for certain parallel block pivot strategies as well. This bridges a theoretical gap, provides solid theoretical basis for parallel implementations of the block algorithm, and provides valuable insights.
Similar content being viewed by others
Notes
Software implementation of the Businger–Golub pivoting is numerically reliable with a modification from [13], which is included in LAPACK starting with the release 3.1.0.
We do so because the absence of the triangular structure causes no additional technical difficulties.
Paige attributed the idea to M. Gentleman.
References
Anderson, E., Bai, Z., Bischof, C., Demmel, J., Dongarra, J., Croz, J.D., Greenbaum, A., Hammarling, S., McKenny, A., Ostrouchov, S., Sorensen, D.: LAPACK Users’ Guide, 2nd edn. SIAM, Philadelphia (1992)
Bai, Z.: Note on the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Linear Algebra Appl. 104, 131–140 (1988)
Barlow, J.: More accurate bidiagonal reduction for computing the singular value decomposition. SIAM J. Matrix Anal. Appl. 23, 761–798 (2002)
Barlow, J., Demmel, J.: Computing accurate eigensystems of scaled diagonally dominant matrices. SIAM J. Numer. Anal. 27(3), 762–791 (1990)
Bečka, M., Okša, G., Vajteršic, M.: Dynamic ordering for a parallel block-Jacobi SVD algorithm. Parallel Comput. 28, 243–262 (2002)
Brent, R.P., Luk, F.T., Van Loan, C.F.: Computation of the singular value decomposition using mesh-connected processors. J. VLSI Comput. Syst. 1(3), 242–270 (1985)
Businger, P.A., Golub, G.H.: Linear least squares solutions by Householder transformations. Numer. Math. 7, 269–276 (1965)
Causey, R.L., Henrici, P.: Convergence of approximate eigenvectors in Jacobi methods. Numer. Math. 2, 67–78 (1960)
Chan, T.F.: An improved algorithm for computing the singular value decomposition. ACM Trans. Math. Softw. 8, 72–83 (1982)
Charlier, J.P., Vanbegin, M., Dooren, P.V.: On efficient implementations of Kogbetliantz’s algorithm for computing the singular value decomposition. Numer. Math. 52, 279–300 (1988)
Demmel, J., Veselić, K.: Jacobi’s method is more accurate than QR. SIAM J. Matrix Anal. Appl. 13(4), 1204–1245 (1992)
Drmač, Z.: A global convergence proof for cyclic Jacobi methods with block rotations. SIAM J. Matrix Anal. Appl. 31, 1329–1350 (2011)
Drmač, Z., Bujanović, Z.: On the failure of rank revealing QR factorization software—a case study. ACM Trans. Math. Softw. 35(2), 1–28 (2008)
Drmač, Z., Veselić, K.: New fast and accurate Jacobi SVD algorithm: I. SIAM J. Matrix Anal. Appl. 29(4), 1322–1342 (2008)
Drmač, Z., Veselić, K.: New fast and accurate Jacobi SVD algorithm: II. SIAM J. Matrix Anal. Appl. 29(4), 1343–1362 (2008)
Fernando, K.V.: Linear convergence of the row-cyclic Jacobi and Kogbetliantz methods. Numer. Math. 56, 71–91 (1989)
Fernando, K.V., Hammarling, S.J.: Kogbetliantz methods for parallel SVD computation: architecture, algorithms and convergence. Tech. Rep. TR9/86, NAG Limited Oxford (1986). (FLAME Working Note 9.)
Forsythe, G.E., Henrici, P.: The cyclic Jacobi method for computing the principal values of a complex matrix. Trans. Am. Math. Soc. 94(1), 1–23 (1960)
Hari, V.: On the quadratic convergence bounds of the serial singular value decomposition by Jacobi methods for triangular matrices. SIAM J. Sci. Stat. Comput. 10, 1076–1096 (1989)
Hari, V., Veselić, K.: On Jacobi methods for singular value decompositions. SIAM J. Sci. Stat. Comput. 8(5), 741–754 (1987)
Hari, V., Zadelj-Martić, V.: Parallelizing the Kogbetliantz method: a first attempt. J. Numer. Anal. Ind. Appl. Math. 2(1), 49–66 (2007)
Kogbetliantz, E.G.: Solution of linear equations by diagonalization of coefficient matrix. Q. Appl. Math. 13, 123–132 (1955)
Luk, F.: A triangular processor array for computing singular values. Linear Algebra Appl. 77, 259–273 (1986)
Luk, F., Park, H.: On parallel Jacobi orderings. SIAM J. Sci. Stat. Comput. 10, 18–26 (1989)
Matejaš, J., Hari, V.: Accuracy of the Kogbetliantz method for scaled diagonally dominant triangular matrices. Appl. Comput. Math. 217(8), 3726–3746 (2010)
von Neumann, J.: On infinite direct products. Compos. Math. 6, 1–77 (1939)
Okša, G., Vajteršic, M.: Special issue: A systolic block-Jacobi SVD solver for processor meshes. Parallel Algorithms Appl. 18(1–2), 49–70 (2003)
Okša, G., Vajteršic, M.: Efficient preprocessing in the parallel block-Jacobi SVD algorithm. Parallel Comput. 31, 166–176 (2005)
Paige, C.C.: Computing the generalized singular value decomposition. SIAM J. Sci. Stat. Comput. 7, 1126–1146 (1986)
Paige, C.C., Dooren, P.V.: On the quadratic convergence of Kogbetliantz’s algorithm for computing the singular value decomposition. Linear Algebra Appl. 7, 301–313 (1986)
Paige, C.C., Wei, M.: History and generality of the CS decomposition. Linear Algebra Appl. 208(209), 303–326 (1994)
Stewart, G.W.: Computing the CS decomposition of a partitioned orthonormal matrix. Numer. Math. 40, 297–306 (1982)
Stewart, G.W.: A Jacobi-like algorithm for computing the Schur decomposition of a non-Hermitian matrix. SIAM J. Sci. Stat. Comput. 6, 853–864 (1985)
Stewart, G.W.: A gap-revealing matrix decomposition. Technical report TR-3771, Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 (1997)
Stewart, G.W.: The QLP approximation to the singular value decomposition. Technical report TR-97-75, Department of Computer Science and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742 (1997)
Stewart, G.W., Sun, J.G.: Matrix Perturbation Theory. Academic Press, San Diego (1990)
Acknowledgement
This work is supported by the Croatian MZOS grant 0372783-2750. The authors would like to thank the anonymous referees for their insightful comments and constructive criticism, which helped in improving this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Daniel Kressner.
Rights and permissions
About this article
Cite this article
Bujanović, Z., Drmač, Z. A contribution to the theory and practice of the block Kogbetliantz method for computing the SVD. Bit Numer Math 52, 827–849 (2012). https://doi.org/10.1007/s10543-012-0388-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10543-012-0388-y