The joint bidiagonalization process with partial reorthogonalization

Abstract

The joint bidiagonalization (JBD) process is a useful algorithm for the computation of the generalized singular value decomposition (GSVD) of a matrix pair. However, it always suffers from rounding errors, which causes the Lanczos vectors to lose their mutual orthogonality. In order to maintain some level of orthogonality, we present a semiorthogonalization strategy. Our rounding error analysis shows that the JBD process with the semiorthogonalization strategy can ensure that the convergence of the computed quantities is not affected by rounding errors and the final accuracy is high enough. Based on the semiorthogonalization strategy, we develop the joint bidiagonalization process with partial reorthogonalization (JBDPRO). In the JBDPRO algorithm, reorthogonalizations occur only when necessary, which saves a big amount of reorthogonalization work, compared with the full reorthogonalization strategy. Numerical experiments illustrate our theory and algorithm.

Appendix: 1: Proofs of Lemma 3.1 and Lemma 3.2

Proof Proof of Lemma 3.1

We prove (3.11) by mathematical induction. For i = 1, from (3.9) and (3.10), we have

$$ \begin{array}{@{}rcl@{}} \hat{\alpha}_{1}{Q_{L}^{T}}\hat{u}_{1} & =& {Q_{L}^{T}}Q_{L}\hat{\nu}_{1} -{Q_{L}^{T}}\hat{f}_{1} \\ & =& (I_{n}-{Q_{A}^{T}}Q_{A})\hat{\nu}_{1} -{Q_{L}^{T}}\hat{f}_{1} \\ & =& \hat{\nu}_{1}-{Q_{A}^{T}}(\alpha_{1}u_{1}+\beta_{2}u_{2}+f_{1})-{Q_{L}^{T}}\hat{f}_{1} \\ & =& \hat{\nu}_{1}-\alpha_{1}(\alpha_{1} \nu_{1}+g_{1})- \beta_{2}(\alpha_{2}\nu_{2}+\beta_{2}\nu_{1}+g_{2}) -{Q_{A}^{T}}f_{1}-{Q_{L}^{T}}\hat{f}_{1} \\ & =& (1-{\alpha_{1}^{2}}-{\beta_{2}^{2}})\hat{\nu}_{1} + \alpha_{2}\beta_{2}\hat{v}_{2} + O(\bar{q}(m,n,p)\varepsilon) . \end{array} $$

Next, suppose (3.11) is true for the indices up to i. For i + 1, we have

$$\hat{\alpha}_{i+1}{Q_{L}^{T}}\hat{u}_{i+1} = {Q_{L}^{T}}Q_{L}\hat{\nu}_{i+1}-\hat{\beta}_{i}{Q_{L}^{T}}\hat{u}_{i}- {\sum}_{j=1}^{i-1}\hat{\xi}_{ji+1}{Q_{L}^{T}}\hat{u}_{j}-{Q_{L}^{T}}\hat{f}_{i+1} .$$

Since $(\hat {\beta }_{i}{Q_{L}^{T}}\hat {u}_{i}- {\sum }_{j=1}^{i-1}\hat {\xi }_{ji+1}{Q_{L}^{T}}\hat {u}_{j}) \in span\{\hat {\nu }_{1}, \dots , \hat {\nu }_{i+1}\}+ O(\bar {q}(m,n,p)\varepsilon )$, we only need to prove ${Q_{L}^{T}}Q_{L}\hat {\nu }_{i+1}\in span\{\hat {\nu }_{1}, \dots , \hat {\nu }_{i+2}\}+ O(\bar {q}(m,n,p)\varepsilon )$. Notice that

$$ \begin{array}{@{}rcl@{}} {Q_{L}^{T}}Q_{L}\hat{\nu}_{i+1} & =& (I_{n}-{Q_{A}^{T}}Q_{A})\hat{\nu}_{i+1} \\ & =& \hat{\nu}_{i+1}+(-1)^{i+1}{Q_{A}^{T}}(\alpha_{i+1}u_{i+1}+\beta_{i+1}u_{i+2}+{\sum}_{j=1}^{i}\xi_{ji+1}u_{j}+f_{i+1}) \end{array} $$

$$ \begin{array}{@{}rcl@{}} \phantom{{Q_{L}^{T}}Q_{L}\hat{\nu}_{i+1}}& =& \hat{\nu}_{i+1}+(-1)^{i+1}(\alpha_{i+1}{Q_{A}^{T}}u_{i+1}+\beta_{i+1}{Q_{A}^{T}}u_{i+2}+ {\sum}_{j=1}^{i}\xi_{ji+1}{Q_{A}^{T}}u_{j}) \\&&+ (-1)^{i+1}{Q_{A}^{T}}f_{i+1} .\end{array} $$

From (3.9), we have

$$ \begin{array}{@{}rcl@{}} (\alpha_{i+1}{Q_{A}^{T}}u_{i+1}+\beta_{i+1}{Q_{A}^{T}}u_{i+2}&+& {\sum}_{j=1}^{i}\xi_{ji+1}{Q_{A}^{T}}u_{j}) \in span\{\hat{\nu}_{1}, \dots, \hat{\nu}_{i+2}\}\\&+& O(\bar{q}(m,n,p)\varepsilon) , \end{array} $$

which completes the proof of the induction step.

By the mathematical induction principle, (3.11) holds for all $i = 1, 2, \dots ,k$. □

Proof Proof of Lemma 3.2

By (3.8) and (3.9), the process of computing U_k+ 1 and V_k can be treated as the Lanczos bidiagonalization of Q_A with the semiorthogonalization strategy. Since the k-step Lanczos bidiagonalization process is equivalent to the (2k + 1)-step symmetric Lanczos process [2, §7.6.1], the bounds for C_k and D_k can be deduced from the property of the symmetric Lanczos process with the semiorthogonalization strategy; see [26, Lemma 4] and its proof.

Now, we give the bound of $\widehat {C}_{k}$. At the (i − 1)-th step, from (3.10), we can write the reorthogonalization step of $\hat {u}_{i}$ as

$$ \begin{array}{@{}rcl@{}} & \hat{\alpha}_{i}^{\prime}\hat{u}_{i}^{\prime} = Q_{L}\hat{\nu}_{i}-\hat{\beta}_{i-1}\hat{u}_{i-1}-\hat{f}_{i}^{\prime}, \end{array} $$

(A.1)

$$ \begin{array}{@{}rcl@{}} & \hat{\alpha}_{i}\hat{u}_{i} = \hat{\alpha}_{i}^{\prime}\hat{u}_{i}^{\prime}- {\sum}_{j=1}^{i-2}\hat{\xi}_{ji}\hat{u}_{j}-\hat{f}_{i}^{\prime\prime} , \end{array} $$

(A.2)

where $\|\hat {f}_{i}^{\prime }\|, \|\hat {f}_{i}^{\prime \prime }\| = O(q_{3}(p,n)\varepsilon )$. Thus, for $l=1,\dots , i-2$, we have

$$\hat{\alpha}_{i}^{\prime}\hat{u}_{l}^{T}\hat{u}_{i}^{\prime} = \hat{u}_{l}^{T}Q_{L}\hat{\nu}_{i}-\hat{\beta}_{i-1}\hat{u}_{l}^{T}\hat{u}_{i-1}- \hat{u}_{l}^{T}\hat{f}_{i}^{\prime} .$$

From (3.11) and its proof, we know that

$$ {Q_{L}^{T}}\hat{u}_{l}={\sum}_{j=1}^{l+1}\lambda_{j}\hat{v}_{j} + O(\bar{q}(m,n,p)\varepsilon) $$

with modest constants λ_j for $j=1,\dots , l+1$. Notice that $\left |\hat {u}_{l}^{T}\hat {u}_{i-1}, \hat {\nu }_{j}^{T}\hat {\nu }_{i}\right | \leq \sqrt {\varepsilon /(2k+1)}$ for $l=1,\dots , i-2$ and $j=1,\dots , l+1$. We obtain

$$ \hat{\alpha}_{i}^{\prime}\hat{u}_{l}^{T}\hat{u}_{i}^{\prime}= {\sum}_{j=1}^{l+1}\lambda_{j}\hat{\nu}_{j}^{T}\hat{\nu}_{i}- \hat{\beta}_{i-1}\hat{u}_{l}^{T}\hat{u}_{i-1} + O(\bar{q}(m,n,p)\varepsilon) =O(\sqrt{\varepsilon}) .$$

Then, we prove $M = \max \limits _{1\leq j \leq i-1}|\hat {\xi }_{ji}|=O(\sqrt {\varepsilon })$. Premultiplying (A.1) by $\hat {u}_{l}^{T}$ and making some arrangement, we obtain

$$\hat{\xi}_{li} = \hat{\alpha}_{i}^{\prime}\hat{u}_{l}^{T}\hat{u}_{i}^{\prime}-\hat{\alpha}_{i}\hat{u}_{l}^{T}\hat{u}_{i}-{\sum}_{j=1,j\neq l}^{i-2}\hat{\xi}_{ji}\hat{u}_{l}^{T}\hat{u}_{j}-\hat{u}_{l}^{T}\hat{f}_{i}^{\prime\prime} .$$

Notice that $\hat {u}_{l}^{T}\hat {u}_{i}=O(\sqrt {\varepsilon })$ and we have proved $\hat {\alpha }_{i}^{\prime }\hat {u}_{l}^{T}\hat {u}_{i}^{\prime }=O(\sqrt {\varepsilon })$ for $l=1,\dots , i-2$. We obtain

$$ |\hat{\xi}_{li}| \leq O(\sqrt{\varepsilon})+O(\sqrt{\varepsilon})+iM\sqrt{\varepsilon}+O(\bar{q}(m,n,p)\varepsilon) . $$

The above right-hand side does not depend on l anymore, and we finally obtain by taking the maximum on the the left-hand side:

$$ (1-i\sqrt{\varepsilon})M \leq O(\sqrt{\varepsilon}) + O(\bar{q}(m,n,p)\varepsilon) . $$

Therefore, we have $M = O(\sqrt {\varepsilon })$. □