Lower bounds on nonnegative rank via nonnegative nuclear norms

Abstract

The nonnegative rank of an entrywise nonnegative matrix $A \in \mathbb {R}^{m \times n}_+$ is the smallest integer $r$ such that $A$ can be written as $A=UV$ where $U \in \mathbb {R}^{m \times r}_+$ and $V \in \mathbb {R}^{r \times n}_+$ are both nonnegative. The nonnegative rank arises in different areas such as combinatorial optimization and communication complexity. Computing this quantity is NP-hard in general and it is thus important to find efficient bounding techniques especially in the context of the aforementioned applications. In this paper we propose a new lower bound on the nonnegative rank which, unlike most existing lower bounds, does not solely rely on the matrix sparsity pattern and applies to nonnegative matrices with arbitrary support. The idea involves computing a certain nuclear norm with nonnegativity constraints which allows to lower bound the nonnegative rank, in the same way the standard nuclear norm gives lower bounds on the standard rank. Our lower bound is expressed as the solution of a copositive programming problem and can be relaxed to obtain polynomial-time computable lower bounds using semidefinite programming. We compare our lower bound with existing ones, and we show examples of matrices where our lower bound performs better than currently known ones.

Appendix: Proof of Proposition 3: Slack matrix of hypercube

In this appendix we prove Proposition 3 concerning the nonnegative rank of the slack matrix of the hypercube. We restate the proposition here for convenience:

Proposition

Let $C_n = [0,1]^n$ be the hypercube in $n$ dimensions and let $S(C_n) \in \mathbb {R}^{2n \times 2^n}$ be its slack matrix. Then

$$\begin{aligned} {{\mathrm{rank}}}_+(S(C_n)) = \left( \frac{\nu _+^{[0]}(S(C_n))}{\Vert S(C_n)\Vert _F}\right) ^2 = 2n. \end{aligned}$$

Proof

The facets of the hypercube $C_n = [0,1]^n$ are given by the linear inequalities $\{x_k \ge 0\}, \; k=1,\ldots ,n$ and $\{x_k \le 1\}, \; k=1,\ldots ,n$, and the vertices of $C_n$ are given by the set $\{0,1\}^n$ of binary words of length $n$. It is easy to see that the slack matrix of the hypercube is a 0/1 matrix: in fact, for a given facet $F$ and vertex $V$, the $(F,V)$’th entry of $S(C_n)$ is given by:

$$\begin{aligned} S(C_n)_{F,V} = \left\{ \begin{array}{ll} 1 &{} \quad \text { if }V \notin F\\ 0 &{} \quad \text { if }V \in F \end{array}\right. . \end{aligned}$$

Since the slack matrix of the hypercube $S(C_n)$ has $2n$ rows we clearly have

$$\begin{aligned} \left( \frac{\nu _+^{[0]}(S(C_n))}{\Vert S(C_n)\Vert _F}\right) ^2 \le {{\mathrm{rank}}}_+(S(C_n)) \le 2n. \end{aligned}$$

To show that we indeed have equality, we exhibit a particular feasible point $W$ of the semidefinite program (3) and we show that this $W$ satisfies $(\langle S(C_n), W \rangle / \Vert S(C_n)\Vert _F)^2$ $= 2n$.

Let

$$\begin{aligned} W = \frac{1}{\sqrt{2^{n-1}}} (2 S(C_n) - J). \end{aligned}$$

(19)

Observe that $W$ is the matrix obtained from $S(C_n)$ by changing the ones into $\frac{1}{\sqrt{2^{n-1}}}$ and zeros into $-\frac{1}{\sqrt{2^{n-1}}}$. For this $W$ we verify using a simple calculation that

$$\begin{aligned} \left( \frac{\langle S(C_n), W \rangle }{\Vert S(C_n)\Vert _F}\right) ^2 = 2n. \end{aligned}$$

It thus remains to prove that $W$ is indeed a feasible point of the semidefinite program (3) and that the matrix

$$\begin{aligned} \begin{bmatrix} I&\quad -W\\ -W^\mathsf T&\quad I\end{bmatrix} \in \mathbb {R}^{(2n + 2^n)\times (2n+2^n)} \end{aligned}$$

can be written as the sum of a nonnegative matrix and a positive semidefinite one. This is the main part of the proof and for this we introduce some notations. Let $\fancyscript{F}$ be the set of facets of the hypercube, and $\fancyscript{V}=\{0,1\}^n$ be the set of vertices. If $F \in \fancyscript{F}$ is a facet of the hypercube, we denote by $\bar{F}$ the opposite facet to $F$ (namely, if $F$ is given by $x_k \ge 0$, then $\bar{F}$ is the facet $x_k \le 1$ and vice-versa). Similarly for a vertex $V \in \fancyscript{V}$ we denote by $\bar{V}$ the opposite vertex obtained by complementing the binary word $V$. Denote by $N_{\fancyscript{F}}:\mathbb {R}^{\fancyscript{F}}\rightarrow \mathbb {R}^{\fancyscript{F}}$ and $N_{\fancyscript{V}}:\mathbb {R}^{\fancyscript{V}}\rightarrow \mathbb {R}^{\fancyscript{V}}$ the “negation” maps, so that we have:

$$\begin{aligned} \forall g \in \mathbb {R}^{\fancyscript{F}},\quad \forall F \in \fancyscript{F},\quad (N_{\fancyscript{F}} g)(F) = g(\bar{F}) \end{aligned}$$

(20)

$$\begin{aligned} \forall h \in \mathbb {R}^{\fancyscript{V}},\quad \forall V \in \fancyscript{V},\quad (N_{\fancyscript{V}} h)(V) = h(\bar{V}) \end{aligned}$$

(21)

Note that in a suitable ordering of the facets and the vertices, the matrix representation of $N_{\fancyscript{F}}$ and $N_{\fancyscript{V}}$ take the following antidiagonal form ($N_{\fancyscript{F}}$ is of size $2n\times 2n$ and $N_{\fancyscript{V}}$ is of size $2^n \times 2^n$):

Consider now the following decomposition of the matrix ${\begin{bmatrix} I&\quad -W\\ -W^\mathsf T&\quad I\end{bmatrix}}$:

$$\begin{aligned} \begin{bmatrix} I&\quad -W\\ -W^\mathsf T&\quad I \end{bmatrix} = \begin{bmatrix} N_{\fancyscript{F}}&\quad 0\\ 0&\quad N_{\fancyscript{V}} \end{bmatrix} + \begin{bmatrix} I-N_{\fancyscript{F}}&\quad -W\\ -W^\mathsf T&\quad I-N_{\fancyscript{V}} \end{bmatrix} \end{aligned}$$

(22)

Clearly the first matrix in the decomposition is nonnegative. The next lemma states that the second matrix is actually positive semidefinite:

Lemma 2

Let $\mathcal F$ and $\mathcal V$ be respectively the set of facets and vertices of the hypercube $C_n=[0,1]^n$ and let $\widehat{W} \in \mathbb {R}^{\fancyscript{F}\times \fancyscript{V}}$ be the matrix:

$$\begin{aligned} \widehat{W}_{F,V} = \left\{ \begin{array}{ll} 1 &{} \quad \text { if }V \notin F\\ -1 &{} \quad \text { if }V \in F \end{array}\right. \;\; \forall F \in \fancyscript{F},\; V \in \fancyscript{V}\end{aligned}$$

Then the matrix

$$\begin{aligned} \begin{bmatrix} I-N_{\fancyscript{F}}&\quad -\gamma \widehat{W}\\ -\gamma \widehat{W}^\mathsf T&\quad I-N_{\fancyscript{V}} \end{bmatrix} \end{aligned}$$

(23)

is positive semidefinite for $\gamma = 1/\sqrt{2^{n-1}}$ (where $N_{\fancyscript{F}}$ and $N_{\fancyscript{V}}$ are defined in (20) and (21)).

Proof

We use the Schur complement to show that the matrix (23) is positive semidefinite. In fact we show that

• 1. $I-N_{\fancyscript{V}} \succeq 0$

• 2. ${{\mathrm{range}}}(\widehat{W}^\mathsf T ) \subseteq {{\mathrm{range}}}(I-N_{\fancyscript{V}})$, and

• 3. $I-N_{\fancyscript{F}} - \gamma ^2 \widehat{W} (I-N_{\fancyscript{V}})^{-1} \widehat{W}^\mathsf T \succeq 0$.

where $(I-N_{\fancyscript{V}})^{-1}$ denotes the pseudo-inverse of $I-N_{\fancyscript{V}}$.

Observe that for any $k \in \mathbb {N}$, the $2k \times 2k$ matrix given by:

is positive semidefinite: in fact one can see that $\frac{1}{2}(I_{2k}-N_{2k})$ is the orthogonal projection onto the subspace spanned by its columns (i.e., the subspace of dimension $k$ spanned by $\{e_i - e_{2k-i}:\;i=1,\ldots ,n\}$ where $e_i$ is the $i$’th unit vector). Hence this shows that $I-N_{\fancyscript{V}}$ is positive semidefinite, and it also shows that $(I-N_{\fancyscript{V}})^{-1} = \frac{1}{4} (I-N_{\fancyscript{V}})$.

Now we show that ${{\mathrm{range}}}(\widehat{W}^\mathsf T ) \subseteq {{\mathrm{range}}}(I-N_{\fancyscript{V}})$. For any $F \in \fancyscript{F}$, the $F$’th column of $\widehat{W}^\mathsf T$ satisfies $(\widehat{W}^\mathsf T )_{V,F} = -(\widehat{W}^\mathsf T )_{\bar{V},F}$ for any $V \in \fancyscript{V}$, and thus ${{\mathrm{range}}}(\widehat{W}^\mathsf T ) \subseteq {{\mathrm{span}}}(e_{V} - e_{\bar{V}}, \; : \; V \in \fancyscript{V}) = {{\mathrm{range}}}(I-N_{\fancyscript{V}})$.

It thus remains to show that

$$\begin{aligned} I-N_{\fancyscript{F}} - \gamma ^2 \widehat{W} (I-N_{\fancyscript{V}})^{-1} \widehat{W}^\mathsf T \succeq 0 \end{aligned}$$

First note that since $\frac{1}{2} (I-N_{\fancyscript{V}})$ is an orthogonal projection and that ${{\mathrm{range}}}(\widehat{W}^\mathsf T ) \subseteq {{\mathrm{range}}}(I-N_{\fancyscript{V}})$, we have $(I-N_{\fancyscript{V}})^{-1} \widehat{W}^\mathsf T = \frac{1}{2} \widehat{W}^\mathsf T$. Thus we now have to show that

$$\begin{aligned} I-N_{\fancyscript{F}} - \frac{\gamma ^2}{2} \widehat{W} \widehat{W}^\mathsf T \succeq 0. \end{aligned}$$

The main observation here is that the matrix $\widehat{W}\widehat{W}^\mathsf T$ is actually equal to $2^{n} (I-N_{\fancyscript{F}})$. For any $F,G \in \fancyscript{F}$, we have:

$$\begin{aligned} (\widehat{W}\widehat{W}^\mathsf T )_{F,G} = \sum \limits _{a \in \fancyscript{V}}\widehat{W}_{F,a} \widehat{W}_{G,a} = \left\{ \begin{array}{ll} 2^n &{} \quad \text { if } F=G\\ -2^n &{} \quad \text { if } F=\bar{G}\\ 0 &{} \quad \text { else} \end{array} \right. \end{aligned}$$

First it is clear that if $F=G$, then $(\widehat{W}\widehat{W}^\mathsf T )_{F,G} = 2^n$. Also if $F=\bar{G}$ then $(\widehat{W}\widehat{W}^\mathsf T )_{F,G} = -2^n$ since if $F=\bar{G}$ then $a \in F \Leftrightarrow a \notin G$ hence $\widehat{W}_{F,a} \widehat{W}_{G,a} = -1$ for all $a \in \fancyscript{V}$. In the case that $F \ne G$ and $F \ne \bar{G}$, it is easy to verify by simple counting that $\sum \nolimits _{a \in \fancyscript{V}}\widehat{W}_{F,a} \widehat{W}_{G,a} = 0$.

Hence we have $I-N_{\fancyscript{F}} - \gamma ^2 \widehat{W} (I-N_{\fancyscript{V}})^{-1} \widehat{W}^\mathsf T = I-N_{\fancyscript{F}} - \frac{\gamma ^2}{2} \widehat{W}\widehat{W}^\mathsf T = (1 - \frac{\gamma ^2}{2}2^n) (I-N_{\fancyscript{F}})$ which is positive semidefinite for $\gamma = 1/\sqrt{2^{n-1}}$.

Using this lemma, Eq. (22) shows that the matrix $W$ is feasible for the semidefinite program (3), and thus that $\nu _+^{[0]}(S(C_n)) \ge \langle S(C_n), W \rangle = \sqrt{2^{n-1}} \cdot 2n$. Hence since $\Vert S(C_n)\Vert _F = \sqrt{2^{n-1} \cdot 2n}$ we get that

$$\begin{aligned} {{\mathrm{rank}}}_+(S(C_n)) \ge \left( \frac{\nu _+^{[0]}(S(C_n))}{\Vert S(C_n)\Vert _F}\right) ^2 \ge \left( \frac{\sqrt{2^{n-1}} \cdot 2n}{\sqrt{2^{n-1} \cdot 2n}}\right) ^2 = 2n \end{aligned}$$

which completes the proof.