Asymptotic quadratic convergence of the parallel block-Jacobi EVD algorithm with dynamic ordering for Hermitian matrices

Abstract

The proof of the asymptotic quadratic convergence is provided for the parallel two-sided block-Jacobi EVD algorithm with dynamic ordering for Hermitian matrices. The discussion covers the case of well-separated eigenvalues as well as clusters of eigenvalues. Having p processors, each parallel iteration step consists of zeroing 2p off-diagonal blocks chosen by dynamic ordering with the aim to maximize the decrease of the off-diagonal Frobenius norm. Numerical experiments illustrate and confirm the developed theory.

Proof of Lemma 1

Proof

The proof of Eq. (6) is similar to the proof of Lemma 1 in [19]. The left-hand side of Eq. (6) is bounded by an intermediate quantity involving $\Vert A_{X_i X_j}\Vert _F$, $\Vert A_{Y_i X_j}\Vert _F$ and $\Vert P_{Y_i X_i}\Vert _2$ as follows (see [19, p. 1076]):

$$\begin{aligned}&\left| \Vert \tilde{A}_{X_i X_j}\Vert _F^2-\Vert A_{X_i X_j}\Vert _F^2 \right| \nonumber \\&\quad \le \Vert P_{Y_i X_i}\Vert _2^2\max \left\{ \Vert A_{X_i X_j}\Vert _F^2,\Vert A_{Y_i X_j}\Vert _F^2\right\} + 2\Vert A_{X_i X_j} \Vert _F\Vert P_{Y_i X_i}\Vert _2\Vert A_{Y_i X_j}\Vert _F. \end{aligned}$$

Then Eq. (6) follows immediately by using $\max \left\{ \Vert A_{X_i X_j}\Vert _F^2,\Vert A_{Y_i X_j}\Vert _F^2\right\} \le \Vert A_{X_1 Y_1}\Vert _F^2$ and $\Vert P_{Y_i X_i}\Vert _2 \le \Vert A_{X_i Y_i}\Vert _F/\delta$. Similarly, the left-hand side of Eq. (7) can be bounded as

$$\begin{aligned}&\left| \Vert \hat{A}_{X_i X_j}\Vert _F^2-\Vert \tilde{A}_{X_i X_j}\Vert _F^2\right| \nonumber \\&\quad \le \Vert P_{Y_j X_j}\Vert _2^2\max \left\{ \Vert \tilde{A}_{X_i X_j}\Vert _F^2, \Vert \tilde{A}_{X_i Y_j}\Vert _F^2\right\} \nonumber \\&\qquad +\, 2\Vert \tilde{A}_{X_i X_j}\Vert _F\Vert P_{Y_j X_j}\Vert _2 \Vert \tilde{A}_{X_i Y_j}\Vert _F. \end{aligned}$$

(31)

To bound the right-hand side, we need to evaluate $\Vert \tilde{A}_{X_i X_j}\Vert _F$ and $\Vert \tilde{A}_{X_i Y_j}\Vert _F$. From the definition of $\tilde{A}_{X_i X_j}$,

$$\begin{aligned} \tilde{A}_{X_i X_j} = P_{X_i X_i}^H A_{X_i X_j}+P_{Y_i X_i}^H A_{Y_i X_j}, \end{aligned}$$

(32)

we have

$$\begin{aligned} \Vert \tilde{A}_{X_i X_j}\Vert _F\le & {} \Vert A_{X_i X_j}\Vert _F+\Vert P_{Y_i X_i}\Vert _2 \Vert A_{Y_iX_j}\Vert _F \nonumber \\\le & {} \Vert A_{X_i X_j}\Vert _F+\frac{\Vert A_{X_i Y_i}\Vert _F}{\delta } \,\Vert A_{X_1 Y_1}\Vert _F. \end{aligned}$$

(33)

On the other hand, from the unitarity of transformation in Eq. (3), we have

$$\begin{aligned} \Vert \tilde{A}_{X_i X_j}\Vert _F^2+\Vert \tilde{A}_{Y_i X_j}\Vert _F^2 = \Vert A_{X_i X_j} \Vert _F^2+\Vert A_{Y_i X_j}\Vert _F^2, \end{aligned}$$

which leads to

$$\begin{aligned} \Vert \tilde{A}_{X_i X_j}\Vert _F^2 \le \Vert A_{X_i X_j}\Vert _F^2+\Vert A_{Y_i X_j}\Vert _F^2 \le 2\Vert A_{X_1 Y_1}\Vert _F^2. \end{aligned}$$

(34)

Similarly,

$$\begin{aligned} \Vert \tilde{A}_{X_i Y_j}\Vert _F^2 \le \Vert A_{X_i Y_j}\Vert _F^2+\Vert A_{Y_i Y_j}\Vert _F^2 \le 2\Vert A_{X_1 Y_1}\Vert _F^2. \end{aligned}$$

(35)

Putting upper bounds from Eqs. (34) and (35) into the first term of Eq. (31), and inserting upper bounds from Eqs. (33) and (35) into the second term of Eq. (31) gives

$$\begin{aligned}&\left| \Vert \hat{A}_{X_i X_j}\Vert _F^2-\Vert \tilde{A}_{X_i X_j}\Vert _F^2\right| \nonumber \\&\quad \le \frac{\Vert A_{X_j Y_j}\Vert _F^2}{\delta ^2}\cdot 2\Vert A_{X_1 Y_1}\Vert _F^2 \nonumber \\&\qquad +\, 2\left( \Vert A_{X_i X_j}\Vert _F+\frac{\Vert A_{X_i Y_i}\Vert _F}{\delta }\, \Vert A_{X_1 Y_1}\Vert _F\right) \cdot \frac{\Vert A_{X_j Y_j}\Vert _F}{\delta }\cdot \sqrt{2}\Vert A_{X_1 Y_1}\Vert _F \nonumber \\&\quad \le 2\,\frac{\Vert A_{X_1 Y_1}\Vert _F^2}{\delta ^2}\left( \Vert A_{X_j Y_j}\Vert _F^2+ \sqrt{2}\,\Vert A_{X_i Y_i}\Vert _F \Vert A_{X_j Y_j}\Vert _F\right) \nonumber \\&\qquad +\,2\sqrt{2}\,\frac{\Vert A_{X_1 Y_1}\Vert _F}{\delta }\,\Vert A_{X_j Y_j}\Vert _F \Vert A_{X_i X_j}\Vert _F \nonumber \\&\quad \le \frac{\Vert A_{X_1 Y_1}\Vert _F^2}{\delta ^2}\left\{ \sqrt{2}\Vert A_{X_i Y_i} \Vert _F^2+(2+\sqrt{2})\Vert A_{X_j Y_j}\Vert _F^2\right\} \nonumber \\&\qquad +\,2\sqrt{2}\,\frac{\Vert A_{X_1 Y_1}\Vert _F}{\delta }\, \Vert A_{X_j Y_j}\Vert _F\Vert A_{X_i X_j}\Vert _F, \end{aligned}$$

where we used the upper bound $2ab\le a^2+b^2$ in the last inequality.

Since the row and column updates for remaining three off-diagonal blocks are, mutatis mutandis block indices, the same as for $A_{X_iX_j}$ (see Eq. (5)), applying the above approach three times one gets remaining upper bounds for the changes of $\Vert A_{X_iY_j}\Vert _F^2$, $\Vert A_{Y_iX_j}\Vert _F^2$ and $\Vert A_{Y_iY_j}\Vert _F^2$. $\square$