Several numerical methods for computing unitary polar factor of a matrix

Abstract

We present several numerical schemes for computing the unitary polar factor of rectangular complex matrices. Error analysis shows high orders of convergence. Many experiments in terms of number of iterations and elapsed times are reported to show the efficiency of the new methods in contrast to the existing ones.

1 Preliminaries

Let $\mathbb{C}^{m\times n}$ ($m\geq n$) denote the linear space of all $m\times n$ complex matrices. The polar decomposition of a complex matrix $A\in\mathbb{C}^{m\times n}$ could be defined as

$$ A=UH,\qquad U^{*}U=I_{r},\qquad \operatorname{rank}(U)=r= \operatorname{rank}(A), $$

(1)

where H is a Hermitian positive semi-definite matrix of order n and $U\in\mathbb{C}^{m\times n}$ is a sub-unitary matrix [1]. A matrix U is sub-unitary if $\|Ux\|_{2}=\|x\|_{2}$ for any $x\in\mathcal{R}(U^{H})=\mathcal{N}(U)^{\bot}$, where $\mathcal{R}$ and $\mathcal{N}$ denote the linear space spanned by columns of matrix X (range of X) and the null space of matrix X, respectively. Note that if $\operatorname{rank}(A)= n$ then $U^{*}U=I_{n}$, and U is an orthonormal Stiefel matrix.

The Hermitian factor H is always unique and can be written as $(A^{*}A)^{\frac{1}{2}}$, while the unitary factor U is unique if A is nonsingular; see for more [2].

It is required to remark that the polar and matrix sign decompositions are intimately connected [3]. For example, Roberts’ integral formula [4],

$$ \operatorname{sign}(A)=\frac{2}{\pi} \int_{0}^{\infty}\bigl(t^{2}I+A^{2} \bigr)^{-1}\, dt, $$

(2)

has an analog in

$$ U=\frac{2}{\pi} \int_{0}^{\infty}\bigl(t^{2}I+A^{*}A \bigr)^{-1}\, dt. $$

(3)

These integral formulas reveal that any property or iterative method involving the matrix sign function can be transformed into one for the polar decomposition by replacing $A^{2}$ via $A^{*}A$, and vice versa.

Practical interest in the polar decomposition stems mainly from the fact that the unitary polar factor of A is the nearest unitary matrix to A in any unitarily invariant norm. The polar decomposition is therefore of interest whenever it is required to orthogonalize a matrix [5]. To obtain more background in this topic, one may refer to [6–9].

Now we briefly review some of the most important iterative matrix methods for computing polar decomposition. Among many iterations (see e.g. [10] and the references therein) available for finding U, the most practically useful one is the Newton iteration. The method of Newton introduced for polar decomposition in [5] is as follows:

$$ U_{k+1}=\frac{1}{2} \bigl(U_{k}+U_{k}^{-*} \bigr), $$

(4)

for the square nonsingular cases and the following alternative for general rectangular cases [11]:

$$ U_{k+1}=\frac{1}{2} \bigl(U_{k}+U_{k}^{\dagger *} \bigr), $$

(5)

wherein $U^{\dagger}$ stands for the Moore-Penrose generalized inverse. Note that, throughout this work, $U_{k}^{-*}$ stands for $(U_{k}^{-1})^{*}$. Similar notations are used as well.

Remark 1.1

We point out that here we focus mainly on computing the unitary polar factor of rectangular matrices, since the high-order methods discussed in this work will not require the computation of pseudo-inverse and is better than the corresponding Newton’s version (5), which requires the computation of one pseudo-inverse per computing cycle.

Recently, an efficient cubically convergent method has been introduced in [12] as follows:

$$ U_{k+1}=U_{k}[38I+42Y_{k}] [9I+60Y_{k}+11Z_{k}]^{-1}, $$

(6)

where $Y_{k}=U_{k}^{*}U_{k}$, $Z_{k}=Y_{k}Y_{k}$.

An (enough close) initial matrix $U_{0}$ must be employed in such matrix fixed-point type methods to ensure convergence. Such an approximation/guess for the unitary factor of the rectangular complex matrices can be constructed by

$$ U_{0}=\frac{1}{\alpha}A, $$

(7)

where $\alpha>0$ is an estimate of $\|A\|_{2}$. This is known as one of the good ways in the literature for constructing an initial value to ensure the convergence of iterative Newton-type methods for finding the unitary polar factor of A.

The other sections of this paper are organized as follows. In Section 2, we derive an iteration function for polar decomposition. Next, Section 3 discusses the convergence properties of this method. It is revealed that the rate of convergence is six since the proposed formulation transforms the singular values of the approximated matrices produced per cycle with a sixth rate to unity (one). This discloses that our method is quite rapid. Several other new iterative methods are constructed in Section 4. Many numerical experiments are provided to support the theoretical aspects of the paper in Section 5. Finally, conclusions are drawn in Section 6.

2 A numerical method

The procedure of constructing a new iterative method for U, is to apply a zero-finder on a particular map [13]. That is, solving the following nonlinear (matrix) equation:

$$ F(U):=U^{*}U-I=0, $$

(8)

where I is the identity matrix, by an appropriate root-finding method could yield novel schemes.

To that end, we first introduce the following iterative expression for finding the simple zeros of nonlinear equations:

$$ \left \{ \textstyle\begin{array}{l} y_{k}=u_{k}-\frac{20-9 L(u_{k})}{20-19 L(u_{k})}\frac {f(u_{k})}{f'(u_{k})}, \\ u_{k+1}=y_{k}-\frac{f(y_{k})}{f'(y_{k})}, \end{array}\displaystyle \right . $$

(9)

with $L(u_{k})=\frac{f''(u_{k}) f(u_{k})}{f'(u_{k})^{2}}$. This is a combination of the cubical method proposed in [12] and the quadratically convergent Newton’s method.

Theorem 2.1

Let $\alpha\in D$ be a simple zero of a sufficiently differentiable function $f:D\subseteq\mathbb{C}\rightarrow\mathbb{C}$ for an open interval D, which contains $x_{0}$ as an initial approximation of α. Then the iterative expression (9) has sixth order of convergence.

Proof

The proof is based on Taylor expansions of the function f around the appropriate points and would be similar to those taken in [14]. As a consequence, it is skipped over. □

Here using (9) for solving $u^{2}-1=0$, we have the following iteration in the reciprocal form:

$$ u_{k+1}=\frac{684 u_{k} + 5\text{,}316 u_{k}^{3} + 5\text{,}876 u_{k}^{5} + 924 u_{k}^{7}}{81 + 2\text{,}524 u_{k}^{2} + 6\text{,}990 u_{k}^{4} + 3\text{,}084 u_{k}^{6} + 121 u_{k}^{8}},\quad k=0,1,\ldots. $$

(10)

The iteration obtained after applying a nonlinear equation solver on the mapping (8) and its reciprocal, could be used for polar decomposition. But here, the experimental results show that the reciprocal form (10) is more stable in the presence of round-off errors.

Drawing the attraction basins [15] of (10) for finding the solution of the polynomial equation $u^{2}-1=0$ in the complex plane reveals that the application of (9) for finding matrix sign function and consequently the unitary polar factor has global convergence. This is done in Figure 1 on the rectangle $[-2,2]\times[-2,2]$.

Figure 1

Attraction basins shaded according to the number of iterations for Newton’s method (left) and ( 10 ) (right), for the polynomial $\pmb{g(u)=u^{2}-1}$ .

By taking into account this global convergence behavior, we extend (10) as follows:

$$\begin{aligned} U_{k+1} =&U_{k}[684 I + 5\text{,}316 Y_{k} + 5\text{,}876 Z_{k} + 924 W_{k}] \\ &{} \times[81I+ 2\text{,}524 Y_{k} + 6\text{,}990 Z_{k} + 3\text{,}084 W_{k} + 121 L_{k}]^{-1}, \end{aligned}$$

(11)

where $U_{0}$ is chosen by (7) (or its simplest form $U_{0}=A$) and $Y_{k}=U_{k}^{*}U_{k}$, $Z_{k}=Y_{k}Y_{k}$, $W_{k}=Y_{k}Z_{k}$, and $L_{k}=Y_{k}W_{k}$. The iteration algorithm (11) converges to the unitary polar factor under some conditions. These discussions will be presented in the next section.

3 Convergence properties

This section is dedicated to the convergence properties of (11) for finding the unitary polar factor of A.

Theorem 3.1

Assume that $A\in\mathbb{C}^{m\times n}$ is an arbitrary matrix. Then the matrix iterates $\{U_{k}\}_{k=0}^{k=\infty}$ of (11) converge to U.

Proof

The proof of this theorem follows the lines of the proofs given in [16]. As such, it is skipped over. □

Theorem 3.2

Let $A\in\mathbb{C}^{m\times n}$ be an arbitrary matrix. Then the new method (11) is of sixth order to find the unitary polar factor of A.

Proof

The proposed scheme (11) transforms the singular values of $U_{k}$ according to the following map:

$$\begin{aligned} \sigma_{i}^{(k+1)} =&{\sigma_{i}^{(k)}} \bigl[684 + 5\text{,}316 {\sigma_{i}^{(k)}}^{2} + 5 \text{,}876 {\sigma_{i}^{(k)}}^{4} + 924 { \sigma_{i}^{(k)}}^{6}\bigr] \\ &{} \times\bigl[81+ 2\text{,}524 {\sigma_{i}^{(k)}}^{2} + 6\text{,}990 {\sigma_{i}^{(k)}}^{4} + 3 \text{,}084 {\sigma_{i}^{(k)}}^{6} + 121 { \sigma_{i}^{(k)}}^{8}\bigr]^{-1}, \end{aligned}$$

(12)

and it leaves the singular vectors invariant. From equation (12), it is enough to show that convergence of the singular values to unity possesses a sixth order of convergence for $k\geq1$. Thus, we arrive at

$$ \frac{\sigma_{i}^{(k+1)}-1}{\sigma_{i}^{(k+1)}+1}= -\frac{(-1+{\sigma_{i}^{(k)}})^{6} (-9+11{\sigma_{i}^{(k)}})^{2}}{(1+{\sigma _{i}^{(k)}})^{6} (9+11{\sigma_{i}^{(k)}})^{2}}. $$

(13)

Taking absolute values from both sides of (13), one gets the following:

$$ \biggl\vert \frac{\sigma_{i}^{(k+1)}-1}{\sigma_{i}^{(k+1)}+1}\biggr\vert \leq \biggl( \frac{-9+11{\sigma_{i}^{(k)}}}{9+11{\sigma_{i}^{(k)}}} \biggr)^{2} \biggl\vert \frac{\sigma_{i}^{(k)}-1}{\sigma_{i}^{(k)}+1}\biggr\vert ^{6}. $$

(14)

This demonstrates the sixth rate of convergence for the proposed numerical algorithm (11). Consequently, the proof is complete. □

Remark 3.1

The presented method is not a member of the Padé family of iterations given in [17] (and discussed deeply in [18]), with global convergence. As a result, it is interesting from both theoretical and computational point of views.

The new formulation (11) is quite rapid, but there is still a way for speeding up the whole process via an acceleration technique given for Newton’s method in [5], known as scaling. Some important scaling approaches were derived in different norms as comes next; we have

$$ \theta_{k}= \biggl(\frac{\|U_{k}^{\dagger}\|_{2}}{\|U_{k}\|_{2}} \biggr)^{\frac{1}{2}}, $$

(15)

where $\|\cdot\|_{2}$ is the spectral norm. This scale factor is optimal in the given $U_{k}$, since (15) minimizes the next error $\| U_{k+1}-U\|_{2}$. Unfortunately, to determine the scale factor (15), one needs to compute two extreme singular values of $U_{k}$ at each iteration. To save the cost of computing the extreme singular values, one might approximate the scaling parameter as in the following [19]:

$$ \theta_{k}= \biggl(\frac{\|U_{k}^{\dagger}\|_{F}}{\|U_{k}\|_{F}} \biggr)^{\frac{1}{2}} $$

(16)

or

$$ \theta_{k}= \biggl(\frac{\|U_{k}^{-1}\|_{1}\|U_{k}^{-1}\|_{\infty}}{\|U_{k}\|_{1}\| U_{k}\|_{\infty}} \biggr)^{\frac{1}{4}}. $$

(17)

Another relatively inexpensive scaling factor is [20]

$$ \theta_{k}=\bigl\vert \det(U_{k})\bigr\vert ^{-1/n}. $$

(18)

The complex modulus of the determinant in this choice is inexpensively obtained from the same matrix factorization used to calculate $U_{k}^{-1}$.

Finally in this section, the new scheme can be expressed in the following accelerated form as well:

$$ \left \{ \textstyle\begin{array}{l} \text{Compute } \theta_{k}\ (\text{for example}) \text{ by (16)},\quad k\geq0, \\ M_{k}=81I+ 2\text{,}524 \theta_{k}^{2}Y_{k} + 6\text{,}990 \theta_{k}^{4}Z_{k} + 3\text{,}084 \theta_{k}^{6}W_{k} + 121 \theta_{k}^{8}L_{k}, \\ U_{k+1}=\theta_{k}U_{k}[684 I + 5\text{,}316 \theta_{k}^{2}Y_{k} + 5\text{,}876 \theta_{k}^{4}Z_{k} + 924 \theta_{k}^{6}W_{k}]M_{k}^{-1}. \end{array}\displaystyle \right . $$

(19)

4 Some other iterative methods

As discussed in the preceding sections, the construction of the iterative methods for finding the unitary polar factor of a matrix mainly relies on the nonlinear equation solver which is going to be applied on the mapping (8).

Now, some may question that the construction (9) is straightforward, since it is the combination of two already known methods. It is here stated that the main goal is to attain a new scheme for a polar decomposition which has global convergence behavior and is new, i.e., it is not a member of the Padé family of iterations (or its reciprocal). So, the novelty and usefulness of (9) in terms of solving nonlinear equations is not of main interest here and the importance is focused on providing a novel and useful scheme for finding the unitary polar factor.

To construct some other new and useful iterative methods for finding the unitary polar factor of a matrix, we could again use the first sub-step of (9) along with different kinds of approximation for the newly appearing first derivative in the second sub-step. As such, we could derive the following nonlinear equation solver:

$$ \left \{ \textstyle\begin{array}{l} y_{k}=u_{k}-\frac{20-9 L(u_{k})}{20-19 L(u_{k})}\frac {f(u_{k})}{f'(u_{k})}, \\ u_{k+1}=y_{k}-\frac{f(y_{k})}{f[u_{k},y_{k}]}, \end{array}\displaystyle \right . $$

(20)

wherein $f[x_{k},y_{k}]$ is the two-point divided difference. Note again that pursuing the optimality conjecture of Kung-Traub or usefulness of the iterative method in terms of solving nonlinear equation is not the only cutting-edge factor, since the most eminent factor is in designing a new scheme for unitary polar factor with global convergence behavior. An application of (20) to equation (8) results in the following fourth-order scheme for the unitary polar factor:

$$ U_{k+1}=U_{k}[47 I + 102 Y_{k} + 11 Z_{k}] [9I+ 98 Y_{k} + 53 Z_{k}]^{-1}. $$

(21)

At this moment, by applying a similar secant-like strategy in a third sub-step after (20), one may design the following seventh-order scheme:

$$ \left \{ \textstyle\begin{array}{l} y_{k}=u_{k}-\frac{20-9 L(u_{k})}{20-19 L(u_{k})}\frac {f(u_{k})}{f'(u_{k})}, \\ z_{k}=y_{k}-\frac{f(y_{k})}{f[u_{k},y_{k}]}, \\ u_{k+1}=z_{k}-\frac{f(z_{k})}{f[z_{k},y_{k}]}, \end{array}\displaystyle \right . $$

(22)

and subsequently the following iterative method:

$$\begin{aligned} U_{k+1} =&U_{k}[765 I + 7\text{,}840 Y_{k} + 12\text{,}866 Z_{k}+4\text{,}008W_{k}+121L_{k}] \\ &{} \times[81I+ 3\text{,}208 Y_{k} + 12\text{,}306 Z_{k}+8\text{,}960W_{k}+1\text{,}045L_{k}]^{-1}. \end{aligned}$$

(23)

The attraction basins of these two new iterative methods are provided in Figure 2, which manifest their global convergence behavior. Note that a theoretical discussion for proving this global behavior is also possible using a similar strategy as given in [16].

Figure 2

Attraction basins shaded according to the number of iterations for ( 20 ) (left) and ( 22 ) (right), for the polynomial $\pmb{g(u)=u^{2}-1}$ .

The error analysis of the new schemes (21) and (23) are similar to the case given in Section 3. As a result, they are not included here.

5 Numerical results

We have tested the contributed methods (11), (21), (23) denoted by PM1, PM2, and PM3, respectively, using the programming package Mathematica 10 in double precision [21]. Apart from this scheme, several iterative methods, such as (5) denoted by NM, and (6) denoted by KHM, and the scaled Newton method (denoted by ANM) are given by

$$ \left \{ \textstyle\begin{array}{l} \text{Compute } \theta_{k} \text{ by (16)},\quad k\geq0, \\ U_{k+1}=\frac{1}{2}[\theta_{k}U_{k}+\theta_{k}^{-1}U_{k}^{\dagger *}], \end{array}\displaystyle \right . $$

(24)

have been tested and compared. We used the following stopping criterion: $R_{k+1}=\frac{\|U_{k+1}-U_{k}\|_{\infty}}{\|U_{k}\|_{\infty}}\leq \epsilon$, wherein $\epsilon=10^{-10}$ is the tolerance.

We now apply different numerical methods for finding the unitary polar factors of many randomly generated rectangular matrices with complex entries. In order to help the readers to re-run the experiments we used $\mathtt{SeedRandom[12345]}$ for producing pseudo-random (complex) numbers.

The random matrices for different dimensions of $m\times n$ are constructed by the following piece of Mathematica code ($I=\sqrt{-1}$):

SeedRandom[12345]; number = 15;

Table[A[l] = RandomComplex[{-10 - 10 I,

                            10 + 10 I}, {m, n}];, {l, number}];

We have gathered up the numerical results for the experiments in Tables 1-6. The initial approximation is constructed as $U_{0}=\frac{1}{\|A\| _{2}}A$. Only for the cases $m\times n=110\times100$ and $m\times n=510\times500$, the comparisons of the required number of iterations have been reported and we mainly focused on the elapsed CPU time (in seconds) to clearly reveal that our proposed scheme is quite efficient in most cases. The results of comparison for the square nonsingular cases of $m\times n=600\times600$ are included in Table 7. This shows that the efficient results are in complete agreement with the CPU time utilized in the execution of program(s) for PM2.

Table 1 Results of comparison for the dimension $\pmb{m\times n=110\times 100}$ in terms of the number of iterations

Table 2 Results of comparison for the dimension $\pmb{m\times n=110\times 100}$ in terms of the elapsed time

Table 3 Results of comparison for the dimension $\pmb{m\times n=210\times 200}$ in terms of the elapsed time

Table 4 Results of comparison for the dimension $\pmb{m\times n=410\times 400}$ in terms of the elapsed time

Table 5 Results of comparison for the dimension $\pmb{m\times n=510\times 500}$ in terms of the number of iterations

Table 6 Results of comparison for the dimension $\pmb{m\times n=510\times 500}$ in terms of the elapsed time

Table 7 Results of comparison for the dimension $\pmb{m\times n=510\times 500}$ in terms of the elapsed time

To give an answer to the key question: whether the increasing order convergence is worth in view of increasing the matrix multiplications in each iteration, it is requisite to incorporate the notion of efficiency index, $p^{1/\theta}$, whereas p and θ stand for the rate of convergence and the computational cots per cycle, respectively. This is achieved by assuming that each matrix-matrix multiplication cost 1-unit while the cost for one regular matrix inverse is 1.5-unit and one matrix Moore-Penrose inverse is 3-unit. Consequently, the efficiency indices for the discussed methods are: $E(\mbox{4})\simeq1.2599$, $E(\mbox{6})\simeq1.2210$, $E(\mbox{11})\simeq1.2698$, $E(\mbox{21})\simeq1.2866$, and $E(\mbox{23})\simeq1.2962$.

However, it is also required to state that for square cases and as could be seen in Table 8, the NM and ANM are better choices since they are using the regular inverses in their iterative structures, unlike their structures in the rectangular cases. Furthermore, it sounds as if the computation of the scaling factor for the proposed method will not be attractive, due to the computation of an extra pseudo-inverse per cycle.

Table 8 Results of comparison for the dimension $\pmb{m\times n=600\times 600}$ in terms of the elapsed time

The acquired numerical results agree with the theoretical discussions given in Sections 2 and 3, overwhelmingly. As a result, we can state that PM1-PM3 reduce the number of iterations and time in finding the polar decomposition.

6 Concluding remarks

In this paper, we developed high-order methods for matrix polar decomposition. It has been shown that the convergence is global. Many numerical tests (of various dimensions) have been provided to show the performance of the new method.

In 1991, Kenney and Laub [17] proposed a family of rational iterative methods for sign (subsequently for polar decomposition), based on Padé approximation. Their principal Padé iterations are convergent globally. Thus, we have convergent methods of arbitrary orders for sign (subsequently for polar decomposition). However, here we tried to propose new methods, which are interesting from theoretical point of view and are not members of Padé family. Numerical results have demonstrated the behavior of the new algorithms.