A Performance Guarantee for Orthogonal Matching Pursuit Using Mutual Coherence

Abstract

In this paper, we present a new performance guarantee for the orthogonal matching pursuit (OMP) algorithm. We use mutual coherence as a metric for determining the suitability of an arbitrary overcomplete dictionary for exact recovery. Specifically, a lower bound for the probability of correctly identifying the support of a sparse signal with additive white Gaussian noise and an upper bound for the mean square error is derived. Compared to the previous work, the new bound takes into account the signal parameters such as dynamic range, noise variance, and sparsity. Numerical simulations show significant improvements over previous work and a much closer correlation to empirical results of OMP.

Appendix

Proof

(proof of Lemma 1) Expanding $\varGamma _j$, we can show that

$$\begin{aligned} \varGamma _j&= |\langle \mathbf {A}_j,\mathbf {A}\mathbf {s}+\mathbf {w}\rangle | \nonumber \\&= \left| \sum _{m=1}^{M}\mathbf {A}_{m,j}\left( \sum _{n=1}^{N}\mathbf {A}_{m,n}\mathbf {s}_n+\mathbf {w}_m\right) \right| \nonumber \\&=\left| \sum _{n=1}^{N}\left\{ \sum _{m=1}^{M}\mathbf {A}_{m,j}\mathbf {A}_{m,n}\mathbf {s}_n + \frac{1}{N}\sum _{m=1}^{M}\mathbf {A}_{m,j}\mathbf {w}_m\right\} \right| . \end{aligned}$$

(30)

Using (5), we have that

$$\begin{aligned} \varGamma _j=\left| \sum _{n=1}^{N}\left\{ \mu _{j,n}\mathbf {s}_n+\frac{1}{N}\langle \mathbf {A}_j,\mathbf {w}\rangle \right\} \right| = \left| \sum \limits _{n=1}^{N}\mathbf {x}_n\right| . \end{aligned}$$

(31)

As mentioned in Sect. 1, we assume that the elements of the sparse vector $\mathbf {s}$ are centered random variables. Hence, the elements of $\mathbf {s}$ are either zero or zero-mean random variables, implying that $\mathrm {E}\{\mathbf {s}_n\}=0$ for all $n=1,\dots ,N$. Together with the fact that $\mathrm {E}\{\mathbf {w}\}=0$, we have

$$\begin{aligned} \mathrm {E}\{\mathbf {x}_n\} = \mu _{j,n}\mathrm {E}\{\mathbf {s}_n\} + N^{-1}\mathrm {E}\{\langle \mathbf {A}_j,\mathbf {w}\rangle \} = 0, \end{aligned}$$

(32)

for all $n=1,\dots ,N$. According to Bernstein’s inequality [2], if $\mathbf {x}_1,\dots , \mathbf {x}_N$ are independent real random variables with mean zero, where $\mathrm {E}\left\{ \mathbf {x}_n^2\right\} \le \nu$, and $\mathrm {Pr}\{|\mathbf {x}_n| < c\}=1$, then

$$\begin{aligned}&\mathrm {Pr}\left\{ \left| \sum \limits _{n=1}^N\mathbf {x}_n\right| \ge \xi \right\} \le 2\;\mathrm {exp}\left( \frac{-\xi ^2}{2\left( \sum \limits _{n=1}^{N}\mathrm {E}\left\{ \mathbf {x}_n^2\right\} +c\xi /3\right) }\right) \nonumber \\&\quad \le 2\;\mathrm {exp}\left( \frac{-\xi ^2}{2(N\nu +c\xi /3)}\right) , \end{aligned}$$

(33)

where (33) follows using (12). This completes the proof.

Proof

(proof of Lemma 2) Equation (13) follows trivially from the triangle inequality. For (14), we have

$$\begin{aligned} \mathrm {E}\left\{ \mathbf {x}_n^2\right\}= & {} \frac{1}{N} \sum _{n=1}^{N} \mathrm {E}\left\{ \mathbf {x}_n^2\right\} \end{aligned}$$

(34)

$$\begin{aligned}= & {} \frac{1}{N} \sum \limits _{n=1}^{N} \mathrm {E}\left\{ \mu _{j,n}^2 \mathbf {s}_n^2 + \frac{1}{N^2}\langle \mathbf {A}_j,\mathbf {w}\rangle ^2 + \frac{2}{N} \mu _{j,n}\mathbf {s}_n\langle \mathbf {A}_j,\mathbf {w}\rangle \right\} \end{aligned}$$

(35)

$$\begin{aligned}= & {} \frac{1}{N} \sum \limits _{n=1}^{N} \mu _{j,n}^2 \mathrm {E}\left\{ \mathbf {s}_n^2\right\} + \frac{1}{N^2}\mathrm {E}\left\{ \langle \mathbf {A}_j,\mathbf {w}\rangle ^2\right\} + \underbrace{\frac{2}{N} \mathrm {E}\left\{ \mu _{j,n}\mathbf {s}_n\langle \mathbf {A}_j,\mathbf {w}\rangle \right\} }_0\qquad \quad \end{aligned}$$

(36)

$$\begin{aligned}\le & {} \frac{\tau }{N} \mu _\mathrm {max}^2 \mathbf {s}_\mathrm {max}^2 + \frac{1}{N^2} \mathrm {E}\left\{ \langle \mathbf {A}_j,\mathbf {w}\rangle ^2\right\} , \end{aligned}$$

(37)

where the last term in (36) is zero since $\mathrm {E}\{\mathbf {w}\}=0$, which implies $\mathrm {E}\{\langle \mathbf {A}_j,\mathbf {w}\rangle \}=0$. Moreover, we have

$$\begin{aligned} \mathrm {E}\left\{ \langle \mathbf {A}_j,\mathbf {w}\rangle ^2\right\}&= \mathrm {E}\left\{ \left( \sum _{m=1}^{M}\mathbf {A}_{m,j}\mathbf {w}_m\right) \left( \sum _{m=1}^{M}\mathbf {A}_{m,j}\mathbf {w}_m\right) \right\} \nonumber \\&= \sum _{m=1}^{M} \mathbf {A}_{m,j}\mathbf {A}_{m,j} \mathrm {E}\{\mathbf {w}_m\mathbf {w}_m\} = \sigma ^2. \end{aligned}$$

(38)

Combining (37) and (38) completes the proof.