Finding sparse solutions of systems of polynomial equations via group-sparsity optimization

Abstract

The paper deals with the problem of finding sparse solutions to systems of polynomial equations possibly perturbed by noise. In particular, we show how these solutions can be recovered from group-sparse solutions of a derived system of linear equations. Then, two approaches are considered to find these group-sparse solutions. The first one is based on a convex relaxation resulting in a second-order cone programming formulation which can benefit from efficient reweighting techniques for sparsity enhancement. For this approach, sufficient conditions for the exact recovery of the sparsest solution to the polynomial system are derived in the noiseless setting, while stable recovery results are obtained for the noisy case. Though lacking a similar analysis, the second approach provides a more computationally efficient algorithm based on a greedy strategy adding the groups one-by-one. With respect to previous work, the proposed methods recover the sparsest solution in a very short computing time while remaining at least as accurate in terms of the probability of success. This probability is empirically analyzed to emphasize the relationship between the ability of the methods to solve the polynomial system and the sparsity of the solution.

Appendices

Appendix 1: Bound on the sparsity level of $\phi$

Lemma 3

For all $(a,b,d) \in (\mathbb {N}^*)^3$ such that $a\ge d(b+d)$, the inequality

$$\begin{aligned} \frac{1}{a}\sum _{q=2}^d \begin{pmatrix}a + q-1\\ q\end{pmatrix} \ge \frac{d}{b} \sum _{q=2}^d\begin{pmatrix}b + q-1\\ q\end{pmatrix} \end{aligned}$$

holds.

Proof

For $q\le d$, we can bound the terms in the sum as

$$\begin{aligned} \frac{1}{a} \begin{pmatrix}a + q-1\\ q\end{pmatrix}&= \frac{(a+q-1)!}{a\, q! (a-1)!} = \frac{1}{a\,q!}\prod _{i=0}^{q-1}(a+i) = \frac{1}{q!}\prod _{i=1}^{q-1}(a+i)\\&\ge \frac{1}{q!} a^{q-1} \\&\ge \frac{1}{q!} d^{q-1}(b+d)^{q-1} \\&\ge \frac{1}{q!} d^{q-1} \prod _{i=1}^{q-1} (b+i) \\&\ge \frac{d^{q-1}}{b} \begin{pmatrix}b + q-1\\ q\end{pmatrix} \end{aligned}$$

where we used $a\ge d(b+d)$ in the second inequality. Then

$$\begin{aligned} \frac{1}{a}\sum _{q=2}^d \begin{pmatrix}a + q-1\\ q\end{pmatrix}&\ge \frac{1}{b} \sum _{q=2}^d d^{q-1} \begin{pmatrix}b + q-1\\ q\end{pmatrix} \\&\ge \frac{d}{b} \sum _{q=2}^d d^{q-2} \begin{pmatrix}b + q-1\\ q\end{pmatrix} \\&\ge \frac{d}{b} \sum _{q=2}^d \begin{pmatrix}b + q-1\\ q\end{pmatrix} \end{aligned}$$

$\square$

Proposition 2

Let the mapping $\phi : \mathbb {R}^n \rightarrow \mathbb {R}^M$ be defined as above. Then, with $d\ge 3$ and $n\ge d(\Vert \varvec{x}\Vert _0 + d)$, the vector $\phi (\varvec{x})$ is sparser than the vector $\varvec{x}$ in the sense that the inequality

$$\begin{aligned} \frac{\Vert \phi (\varvec{x})\Vert _0}{M} \le \frac{2 \Vert \varvec{x}\Vert _0 }{d n} \end{aligned}$$

holds for all $\varvec{x}\in \mathbb {R}^n$.

Proof

By construction, the number of nonzeros in $\phi (\varvec{x})$ is equal to the sum over $q$, $1\le q\le d$, of the number of monomials of degree $q$ in $\Vert \varvec{x}\Vert _0$ variables:

$$\begin{aligned} \frac{\Vert \phi (\varvec{x})\Vert _0}{M}&= \frac{ \sum _{q=1}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{ \sum _{q=1}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} = \frac{ \Vert \varvec{x}\Vert _0 }{n} \frac{ \frac{1}{\Vert \varvec{x}\Vert _0 } \sum _{q=1}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{ \frac{1 }{n} \sum _{q=1}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} \\&= \frac{ \Vert \varvec{x}\Vert _0 }{n} \frac{ 1 + \frac{1}{\Vert \varvec{x}\Vert _0 } \sum _{q=2}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{1+ \frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} \\&= \frac{ \Vert \varvec{x}\Vert _0 }{n} \left[ \frac{ 1}{1+ \frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} + \frac{ \frac{1}{\Vert \varvec{x}\Vert _0 } \sum _{q=2}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{1+ \frac{1 }{n}\sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}}\right] \end{aligned}$$

The assumption $n\ge d(\Vert \varvec{x}\Vert _0 + d)$ implies that⁶ $d\le n$. With $d\le n$, we have

$$\begin{aligned} \frac{1 }{n} \begin{pmatrix}n + q-1\\ q\end{pmatrix} \ge \frac{1}{q!} n^{q-1} \ge \frac{1}{q!} d^{q-1} \end{aligned}$$

which yields

$$\begin{aligned} \frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix} \ge \frac{1 }{n}\sum _{q=2}^d \begin{pmatrix}n + 2-1\\ 2\end{pmatrix} \ge (d-1)\frac{d}{2} \end{aligned}$$

Now, on the one hand we have

$$\begin{aligned} d\ge 3 \Rightarrow \,&\frac{1}{2}d^2 -\frac{3}{2}d + 1 \ge 0 \\ \Rightarrow \,&1 + \frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix} \ge d \\ \Rightarrow \,&\frac{ 1}{1+ \frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} \le \frac{1}{d} \end{aligned}$$

and on the other hand, Lemma 3 yields

$$\begin{aligned} \frac{ \frac{1}{\Vert \varvec{x}\Vert _0 } \sum _{q=2}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{1+\frac{1 }{n} \sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} \le \frac{ \frac{1}{\Vert \varvec{x}\Vert _0 } \sum _{q=2}^d\begin{pmatrix}\Vert \varvec{x}\Vert _0 + q-1\\ q\end{pmatrix}}{\frac{1 }{n}\sum _{q=2}^d\begin{pmatrix}n + q-1\\ q\end{pmatrix}} \le \frac{1}{d} \end{aligned}$$

Thus,

$$\begin{aligned} \frac{\Vert \phi (\varvec{x})\Vert _0}{M} \le \frac{2 \Vert \varvec{x}\Vert _0 }{d n} \end{aligned}$$

$\square$

Appendix 2: Other conditions for sparse recovery

The following uses the exact value of $\Vert \varvec{\phi }_0\Vert _0$.

Theorem 7

Let $\varvec{x}_0$ denote the unique solution to (1)–(2). If the inequality

$$\begin{aligned} \sum _{q=1}^d\begin{pmatrix}\Vert \varvec{x}_0\Vert _0 + q-1\\ q\end{pmatrix} \le \frac{1}{2}\left( 1+ \frac{1}{\mu (\varvec{A})}\right) \end{aligned}$$

(33)

holds, then the solution $\hat{\varvec{\phi }}$ to (5) is unique and equal to $\phi (\varvec{x}_0)$, thus providing $\hat{\varvec{x}} = \varvec{x}_0$.

Another more compact but slightly less tight result is as follows.

Theorem 8

Let $\varvec{x}_0$ denote the unique solution to (1)–(2). If the inequality

$$\begin{aligned} \begin{pmatrix}\Vert \varvec{x}_0\Vert _0 + d-1\\ d\end{pmatrix} \le \frac{1}{2d}\left( 1+ \frac{1}{\mu (\varvec{A})}\right) \end{aligned}$$

(34)

holds, then the solution $\hat{\varvec{\phi }}$ to (5) is unique and equal to $\phi (\varvec{x}_0)$, thus providing $\hat{\varvec{x}} = \varvec{x}_0$.

Proof

Since the terms in the sum of Theorem 7 form an increasing sequence, we have

$$\begin{aligned} \sum _{q=1}^d\begin{pmatrix}\Vert \varvec{x}_0\Vert _0 + q-1\\ q\end{pmatrix} \le d \begin{pmatrix}\Vert \varvec{x}_0\Vert _0 + d-1\\ d\end{pmatrix} \end{aligned}$$

which yields the sought statement by application of Theorem 7. $\square$

Appendix 3: Value of $m$

Let us define $m$ as the number of monomials involving a base variable $x$ with a degree $\ge 1$. It can be computed as the sum over $q,\, 0\le q\le d-1$ of the number of monomials of degree $q$ in $n-1$ variables times the remaining degree $d-q$ (since each monomial in $n-1$ variables can be multiplied by $x$ or $x^2 \ldots$ or $x^{d-q}$ to produce a monomial of degree at most $d$ in $n$ variables):

$$\begin{aligned} m = \sum _{q=0}^{d-1} (d-q)\begin{pmatrix}(n-1) + q-1\\ q\end{pmatrix} = \sum _{q=0}^{d-1} (d-q)\begin{pmatrix}n + q-2\\ q\end{pmatrix} . \end{aligned}$$

Another technique computes $m$ as the total number of all monomials minus the number of monomials not involving $x$ which is the number of monomials in $n-1$ variables:

$$\begin{aligned} m&= M - \sum _{q=1}^{d} \begin{pmatrix}n + q-2\\ q\end{pmatrix} \\&= \sum _{q=1}^{d}\begin{pmatrix}n + q-1\\ q\end{pmatrix} - \begin{pmatrix}n + q-2\\ q\end{pmatrix} \\&= \sum _{q=1}^{d} \left[ \frac{(n + q-1)}{n-1} -1\right] \begin{pmatrix}n + q-2\\ q\end{pmatrix}\\&=\sum _{q=1}^{d} \frac{q}{n-1} \begin{pmatrix}n + q-2\\ q\end{pmatrix} . \end{aligned}$$