Robust Principal Component Pursuit via Inexact Alternating Minimization on Matrix Manifolds

Abstract

Robust principal component pursuit (RPCP) refers to a decomposition of a data matrix into a low-rank component and a sparse component. In this work, instead of invoking a convex-relaxation model based on the nuclear norm and the $\ell ^1$-norm as is typically done in this context, RPCP is solved by considering a least-squares problem subject to rank and cardinality constraints. An inexact alternating minimization scheme, with guaranteed global convergence, is employed to solve the resulting constrained minimization problem. In particular, the low-rank matrix subproblem is resolved inexactly by a tailored Riemannian optimization technique, which favorably avoids singular value decompositions in full dimension. For the overall method, a corresponding $q$-linear convergence theory is established. The numerical experiments show that the newly proposed method compares competitively with a popular convex-relaxation based approach.

Appendix: Local Convergence of an Alternating Projection Method.

Here we consider a heuristic alternating projection method for the RPCP problem. This method, which can be interpreted as an exact alternating minimizer scheme for the optimization problem (1) with $\mu =0$, can be shortly described as follows. Given $A^k\in \mathcal {M}$, one generates

$$\begin{aligned} \left\{ \begin{array}{l} B^{k+1}:=P_\mathcal {N}(Z-A^k), \\ A^{k+1}:=P_\mathcal {M}(Z-B^{k+1}). \end{array}\right. \end{aligned}$$

(24)

The name “alternating projection method” is termed, since the iterative procedure (on $\{A^k\}$) can be expressed as

$$\begin{aligned} A^{k+1}=\psi (A^k):=(P_\mathcal {M}\circ \iota \circ P_\mathcal {N}\circ \iota )(A^k), \end{aligned}$$

(25)

with $\iota :A\mapsto Z-A$, and thus generalizes the classical alternating projection (where $\iota$ is the identity map) in, e.g, [21]. The following theorem asserts the local convergence of the alternating projection method. However, we note that the global convergence for this method is not guaranteed in general.

Theorem 4.1

Given $A^0\in \mathcal {M}$, let the sequence $\{(A^k,B^k)\}$ be iteratively generated by formula (24). Assume that $(A^k,B^k)$ is sufficiently close to some $(A^*,B^*)$ such that $\mathrm{rank}(A^*)=r$, $\Vert B^*\Vert =s$, $T_\mathcal {M}(A^*)\cap T_\mathcal {N}(B^*)=\{0\}$, and moreover

$$\begin{aligned} \left\{ \begin{array}{l} B^*:=P_\mathcal {N}(Z-A^*), \\ A^*:=P_\mathcal {M}(Z-B^*). \end{array}\right. \end{aligned}$$

(26)

Then $\{(A^k,B^k)\}$ converges to $(A^*,B^*)$ $q$-linearly at rate $\kappa _p$; i.e.

$$\begin{aligned} \limsup _{k\rightarrow \infty }\frac{\Vert (A^{k+1},B^{k+1})-(A^*,B^*)\Vert }{\Vert (A^k,B^k)-(A^*,B^*)\Vert }\le \kappa _p, \end{aligned}$$

where $\kappa _p\in [0,1)$ is a constant (same as in Lemma 3.10) such that

$$\begin{aligned} \Vert (P_{T_\mathcal {M}(A^*)}\circ P_{T_\mathcal {N}(B^*)})(\Delta )\Vert \le \kappa _p\Vert \Delta \Vert , \end{aligned}$$

for all $\Delta \in \mathbb {R}^{m\times n}$.

Proof

We only prove the $q$-linear convergence on $\{A^k\}$, as the proof for $\{B^k\}$ is almost identical. Note that $\mathcal {M}$ and $\mathcal {N}$ are two smooth manifolds near $A^*$ and $B^*$, respectively. For the existence of a qualified constant $\kappa _p$, we refer to Lemma 3.10(iii).

In the following, we perturb both equations in (26) with respect to $A^*$ by an arbitrarily fixed $\Delta \in \mathbb {R}^{m\times n}$. The perturbation of the first equation gives

$$\begin{aligned} P_\mathcal {N}(Z-A^*-\Delta )&= B^*+P_\mathcal {N}(Z-A^*-\Delta )\nonumber \\&-P_\mathcal {N}(Z-A^*)\\&= B^*+P_{T_\mathcal {N}(B^*)}(-\Delta )+O(\Vert \Delta \Vert ^2). \end{aligned}$$

Since $A^*$ is a fixed point of the map $\psi$ in (25), the second equation in (26) can be written as $\psi (A^*)=P_\mathcal {M}(Z-B^*)$. Then we have

$$\begin{aligned} \psi (A^*+\Delta )&=P_\mathcal {M}(Z-P_\mathcal {N}(Z-A^*-\Delta ))\\&=P_\mathcal {M}(Z-B^*-P_{T_\mathcal {N}(B^*)}(-\Delta )+O(\Vert \Delta \Vert ^2)) \\&=A^*+P_\mathcal {M}(Z-B^*-P_{T_\mathcal {N}(B^*)}(-\Delta )\\&\quad +O(\Vert \Delta \Vert ^2))-P_\mathcal {M}(Z-B^*) \\&=A^*+(P_{T_\mathcal {M}(A^*)}\circ P_{T_\mathcal {N}(B^*)})(\Delta )+O(\Vert \Delta \Vert ^2)). \end{aligned}$$

Thus, by considering $\Delta =A^k-A^*$ and passing $\Delta \rightarrow 0$, we conclude that

$$\begin{aligned} \limsup _{k\rightarrow \infty }\frac{\Vert A^{k+1}-A^*\Vert }{\Vert A^k-A^*\Vert }\le \kappa _p. \end{aligned}$$

$\square$