A global exact penalty for rank-constrained optimization problem and applications

Abstract

This paper considers a rank-constrained optimization problem where the objective function is continuously differentiable on a closed convex set. After replacing the rank constraint by an equality of the truncated difference of L1 and L2 norm, and adding the equality constraint into the objective to get a penalty problem, we prove that the penalty problem is exact in the sense that the set of its global (local) optimal solutions coincides with that of the original problem when the penalty parameter is over a certain threshold. This establishes the theoretical guarantee for the truncated difference of L1 and L2 norm regularization optimization including the work of Ma et al. (SIAM J Imaging Sci 10(3):1346–1380, 2017). Besides, for the penalty problem, we propose an extrapolation proximal difference of convex algorithm (epDCA) and prove the sequence generated by epDCA converges to a stationary point of the penalty problem. Further, an adaptive penalty method based on epDCA is constructed for the original rank-constrained problem. The efficiency of the algorithms is verified via numerical experiments for the nearest low-rank correlation matrix problem and the matrix completion problem.

Appendices

Appendix 1: The proof of the Lemma 4.1

Proof

(a) Fix any $\alpha \in \mathbb {R}$. Consider the level set $\ell _\alpha (g):=\{x\in \mathbb {R}^d \ |\ g(x)\le \alpha \}$. Notice that

$$\begin{aligned} \ell _\alpha (g)=\{(x,t)\in \mathbb {R}^d\times \mathbb {R}\ |\ g(x)\le t\} g\cap \{(x,t)\in \mathbb {R}^d\times \mathbb {R}\ |\ t=\alpha \}. \end{aligned}$$

Since g and $\{(x,t)\in \mathbb {R}^d\times \mathbb {R}\ |\ t=\alpha \}$ are semialegbraic, we conclude that $\ell _\alpha (g)$ is semialegbraic.

(b) By the definition, $\Vert X\Vert$ can be written as

$$\begin{aligned} \Vert X\Vert =\sup _{u\in \mathbb {R}^{m\times 1},v\in \mathbb {R}^{n\times 1}}\{\langle X,uv^T\rangle s.t.\Vert u\Vert =1,\Vert v\Vert =1\},\ \ \ X\in \mathbb {R}^{m\times n}. \end{aligned}$$

Since Euclidean unit ball is a semialgebraic set [2], it holds that spectral norm $\Vert X\Vert$ is a semialgebraic.

Next, we prove $\Vert \cdot \Vert _h$ is semialgebraic. Since composition of semialgebraic functions are semialgebraic and $\Vert \cdot \Vert _h:=h(\sigma (\cdot ))$, we only need to show that the function h is semialgebraic. Let $h_1(x):={\textstyle \sum _{i=1}^{r-1}}|x|_i^{\downarrow }$ and $h_2(x):=\sqrt{{\textstyle \sum _{i=r}^q}[|x|_i^{\downarrow }]^2}$. Notice that finite sums of semialgebraic functions are semialgebraic. Thus, it suffices to prove that $h_1(x)$ and $h_2(x)$ are semialgebraic. For a positive integer q, we let $S_q$ denote the set of all possible permutations of $\{1, \ldots , q\}$. For each fixed permutation $\sigma \in S_q$, define

$$\begin{aligned} \Omega _\sigma :=\{x\in \mathbb {R}^q \ | \ |x_{\sigma (1)}|\ge |x_{\sigma (2)}|\ge \cdots \ge |x_{\sigma (q)}| \}, \end{aligned}$$

then we have $\bigcup _{\sigma \in S_q} \Omega _\sigma = \mathbb {R}^q$. Thus, for any $x\in \mathbb {R}^q$, there exists a permutation $\sigma \in S_q$ such that $x\in \Omega _\sigma$, $h_1(x)={\textstyle \sum _{i=1}^{r-1}} |x_{\sigma (i)}|$ and $h_2(x)=\sqrt{{\textstyle \sum _{i=r}^q}x_{\sigma (i)}^2}$. The graph of $h_1(x)$ is the set

$$\begin{aligned} \bigcup _{\sigma \in S_q} \{ (x,t)\in \mathbb {R}^q \times \mathbb {R}_+ \ | \ {\textstyle \sum _{i=1}^{r-1}} |x_{\sigma (i)}|= t\}. \end{aligned}$$

Since the set $\{ (x,t)\in \mathbb {R}^q \times \mathbb {R}_+ \ | \ {\textstyle \sum _{i=1}^{r-1}} |x_{\sigma (i)}|= t\}$ is semialgebraic and the number of $S_q$ is finite. Hence the graph of $h_1(x)$ is semialgebraic, and then the function $h_1(x)$ is semialgebraic. Next, we consider the graph of $h_2(x)$,

$$\begin{aligned} \bigcup _{\sigma \in S_q} \{ (x,t)\in \mathbb {R}^q \times \mathbb {R}_+ \ | \ \sqrt{{\textstyle \sum _{i=r}^q} x_{\sigma (i)}^2 } = t\} = \bigcup _{\sigma \in S_q} \{ (x,t)\in \mathbb {R}^q \times \mathbb {R}_+ \ | \ {\textstyle \sum _{i=r}^q} x_{\sigma (i)}^2 = t^2\}. \end{aligned}$$

Using a similar argument as above, one can see that the function $h_2(x)$ is semialgebraic.

For the function $(\Vert \cdot \Vert _h)^*$, we have by the definition of conjugate function

$$\begin{aligned} (\Vert \cdot \Vert _h)^*(W)=\sup _{V\in \mathbb {R}^{m\times n}}\{\langle V,W \rangle -\Vert V\Vert _h\}=\delta _\mathbb {B}(W), \ \ \forall W\in \mathbb {R}^{m\times n} \end{aligned}$$

where $\mathbb {B}:=\{Z\in \mathbb {R}^{m\times n} \ | \ \Vert Z\Vert _h{_*}\le 1\}$ denotes the unit ball on $\Vert \cdot \Vert _h{_*}$. Notice that $\Vert \cdot \Vert _h{_*}$ is the dual norm of $\Vert \cdot \Vert _h$ and $\Vert \cdot \Vert _{h^*}=\sup _{\Vert Z\Vert _h\le 1}\langle \cdot ,Z\rangle .$ From Lemma 4.1 (a) and the result that $\Vert \cdot \Vert _h$ is semialgebraic, we get that $\{Z\in \mathbb {R}^{m\times n}\ | \ \Vert Z\Vert _h\le 1\}$ is a semialgebraic set, which implies the dual norm $\Vert \cdot \Vert _{h^*}$ is a semialgebraic function and then $\mathbb {B}$ is a semialgebraic set. By the fact that indicator function of semialgebraic set is semialgebraic function [2], we get the result that $(\Vert \cdot \Vert _h)^*$ is semialgebraic.

Therefore, the proof is complete. $\square$

Appendix 2: The proof of the Lemma 4.2

Proof

From [2], the correlation matrix set is semialgebraic. Since $\Vert \cdot \Vert$ is a semialgebraic function, then its level set is semialgebraic. The density set $\{X\in \mathbb {S}^{n}_{+} \ | \ tr(X)=1\}$ is equivalent to $\{X\in \mathbb {S}^{n}_{+} \ | \ \Vert X\Vert _{*}=1\}$, and the set $\{X\in \mathbb {S}^{n}_{+} |X\preceq I\}$ is equivalent to $\{X\in \mathbb {S}^{n}_{+} |\Vert X\Vert \le 1\}$, and thus these sets are semialgebraic. The proof is complete. $\square$