Super-Resolution from Noisy Data

Abstract

This paper studies the recovery of a superposition of point sources from noisy bandlimited data. In the fewest possible words, we only have information about the spectrum of an object in the low-frequency band [−f _lo,f _lo] and seek to obtain a higher resolution estimate by extrapolating the spectrum up to a frequency f _hi>f _lo. We show that as long as the sources are separated by 2/f _lo, solving a simple convex program produces a stable estimate in the sense that the approximation error between the higher-resolution reconstruction and the truth is proportional to the noise level times the square of the super-resolution factor (SRF) f _hi/f _lo.

Appendices

Appendix A: Proof of Lemma 2.5

We use the construction described in Sect. 2 of [6]. In more detail,

$$ q(t) = \sum_{t_k \in T} \alpha_k G(t-t_k) + \beta_k G^{ ( 1 ) }(t-t_k), $$

where $\alpha, \beta\in\mathbb{C}^{\vert T \vert }$ are coefficient vectors,

$$ G(t) = \biggl[\frac{\sin ( ( \frac{f_{\mathrm{lo}}}{2}+1 ) \pi t ) }{ ( \frac{f_{\mathrm{lo}}}{2}+1 ) \sin ( \pi t ) } \biggr]^4, \quad t \in\mathbb{T} \setminus\{0\}, $$

(A.1)

and G(0)=1; here, G ^(ℓ) is the ℓth derivative of G. If f _lo is even, G(t) is the square of the Fejér kernel. By construction, the coefficients α and β are selected such that for all t _j ∈T,

$$\begin{aligned} q(t_j) & = v_j \\ q'(t_j) & = 0. \end{aligned}$$

Without loss of generality we consider t _j =0 and bound q(t)−v _j in the interval [0,0.16λ _lo]. To ease notation, we define w(t)=q(t)−v _j =w _R (t)+iw _I (t), where w _R is the real part of w and w _I the imaginary part. Leveraging different results from Sect. 2 in [6] (in particular the equations in (2.25) and Lemmas 2.2 and 2.7), we have

$$\begin{aligned} \bigl \vert w_R'' ( t ) \bigr \vert & = \biggl \vert \sum_{t_k \in T}\operatorname{Re} ( \alpha_k ) G ^{ ( 2 ) } ( t-t_k ) + \sum _{t_k \in T}\operatorname{Re} ( \beta_k ) G ^{ ( 3 ) } ( t-t_k ) \biggr \vert \\ & \leq\| \alpha \| _{L_\infty}\sum_{t_k \in T}\bigl \vert G^{ ( 2 ) } ( t-t_k ) \bigr \vert + \| \beta \| _{L_\infty}\sum_{t_k \in T} \bigl \vert G^{ ( 3 ) } ( t-t_k ) \bigr \vert \\ & \leq C_{\alpha} \biggl( \bigl \vert G^{ ( 2 ) } ( t ) \bigr \vert + \sum_{t_k \in T\setminus\{0\}} \bigl \vert G^{ ( 2 ) } ( t-t_k ) \bigr \vert \biggr) \\ &\quad\ \ {}+ C_{\beta}\lambda_{\mathrm{lo}} \biggl( \bigl \vert G^{ ( 3 ) } ( t ) \bigr \vert + \sum _{t_k \in T\setminus\{0\}} \bigl \vert G^{ ( 3 ) } ( t-t_k ) \bigr \vert \biggr) \\ & \leq C f_{\mathrm{lo}}^2. \end{aligned}$$

The same bound holds for w _I . Since w _R (0), $w_{R}'(0)$, w _I (0) and $w_{I}'(0)$ are all equal to zero, this implies $\vert w_{R}(t) \vert \leq C' f_{\mathrm{lo}}^{2} t^{2}$ and $\vert w_{I}(t) \vert \leq C' f_{\mathrm {lo}}^{2} t^{2}$ in the interval of interest, which allows the conclusion

$$\begin{aligned} \bigl \vert w(t) \bigr \vert \leq C f_{\mathrm{lo}}^2 t^2. \end{aligned}$$

Appendix B: Proof of Lemma 2.7

The proof is similar to that of Lemma 2.4 (see Sect. 2 of [6]), where a low-frequency kernel and its derivative are used to interpolate an arbitrary sign pattern on a support satisfying the minimum-distance condition. More precisely, we set

$$ q_1(t) = \sum_{t_k \in T} \alpha_k G(t-t_k) + \beta_k G ^{ ( 1 ) }(t-t_k), $$

(B.1)

where $\alpha, \beta\in\mathbb{C}^{\vert T \vert }$ are coefficient vectors, G is defined by (A.1). Note that G, G ⁽¹⁾ and, consequently, q ₁ are trigonometric polynomials of degree at most f ₀. By Lemma 2.7 in [6], it holds that for any t ₀∈T and $t \in\mathbb{T}$ obeying |t−t ₀|≤0.16λ _lo,

$$\begin{aligned} \sum_{t_k \in T\setminus\{t_0\}} \bigl \vert G^{ ( \ell ) } ( t-t_k ) \bigr \vert \leq C_{\ell} f_{\mathrm{lo}}^{\ell}, \end{aligned}$$

(B.2)

where C _ℓ is a positive constant for ℓ=0,1,2,3; in particular, C ₀≤0.007, C ₁≤0.08 and C ₂≤1.06. In addition, there exist other positive constants $C_{0}'$ and $C_{1}'$, such that for all t ₀∈T and $t\in\mathbb{T}$ with |t−t ₀|≤Δ/2,

$$\begin{aligned} \sum_{t_k \in T\setminus\{t_0\}} \bigl \vert G^{ ( \ell ) } ( t-t_k ) \bigr \vert \leq C_{\ell}' f_{\mathrm{lo}}^{\ell} \end{aligned}$$

(B.3)

for ℓ=0,1. We refer to Sect. 2.3 in [6] for a detailed description of how to compute these bounds.

In order to satisfy (2.18) and (2.19), we constrain q ₁ as follows: for each t _j ∈T,

$$\begin{aligned} q_1(t_j) & = 0, \\ q_1'(t_j) & = v_j. \end{aligned}$$

Intuitively, this forces q ₁ to approximate the linear function v _j (t−t _j ) around t _j . These constraints can be expressed in matrix form,

$$\left [\begin{array}{c@{\quad}c} D_0 & D_1\\ D_1 & D_2 \end{array} \right ] \begin{bmatrix} \alpha\\ \beta \end{bmatrix} = \begin{bmatrix} 0\\ v \end{bmatrix} , $$

where

$$( D_0 ) _{jk} = G ( t_j - t_k ) , \quad\quad ( D_1 ) _{jk} = G^{ ( 1 ) } ( t_j - t_k ) , \quad\quad ( D_2 ) _{jk} = G ^{ ( 2 ) } ( t_j - t_k ) , $$

and j and k range from 1 to |T|. It is shown in Sect. 2.3.1 of [6] that under the minimum-separation condition this system is invertible, so that α and β are well defined. These coefficient vectors can consequently be expressed as

$$\begin{bmatrix} \alpha\\ \beta \end{bmatrix} = \begin{bmatrix} -D_0^{-1}D_1 \\ {I} \end{bmatrix} S^{-1} v,\quad \quad S := D_2-D_1D_0^{-1}D_1, $$

where S is the Schur complement. Inequality (B.2) implies

$$\begin{aligned} \| {I}- D_0 \| _{\infty} & \le C_0 , \end{aligned}$$

(B.4)

$$\begin{aligned} \| D_1 \| _{\infty} & \le C_{1} f_{\mathrm {lo}}, \end{aligned}$$

(B.5)

$$\begin{aligned} \| \kappa{I}-D_2 \| _{\infty} & \le C_{2} f_{\mathrm{lo}}^2 , \end{aligned}$$

(B.6)

where κ=|G ⁽²⁾(0)|=π ² f _lo(f _lo+4)/3.

Let ∥M∥_∞ denote the usual infinity norm of a matrix M defined as $\|M\|_{\infty}= \max_{\|x\|_{\infty}= 1} \|Mx\|_{\infty}= \max_{i} \sum_{j} |a_{ij}|$. Then, if ∥I−M∥_∞<1, the series M ⁻¹=(I−(I−M))⁻¹=∑ _k≥0(I−M)^k is convergent and we have

$$ \| M^{-1} \| _{\infty} \leq\frac {1}{1-\| {I}-M \| _{\infty}}. $$

This, together with (B.4), (B.5) and (B.6) implies

$$\begin{aligned} \| D_0^{-1} \| _{\infty} & \leq\frac {1}{1-\| {I}-D_0 \| _{\infty}} \leq \frac{1}{1-C_0} , \\ \| \kappa{I}- S \| _{\infty} & \leq \| \kappa{I} -D_2 \| _{\infty}+\| D_1 \| _{\infty}\| D_0^{-1} \| _{\infty}\| D_1 \| _{\infty} \leq \biggl( C_2+\frac{C_1^2}{1-C_0} \biggr) f_{\mathrm{lo}}^2 , \\ \| S^{-1} \| _{\infty} & = \kappa^{-1} \biggl\| \biggl( \frac {S}{\kappa} \biggr) ^{-1} \biggr\|_{\infty} \leq\frac{1}{\kappa-\| \kappa{I}-S \| _{\infty}} \leq \biggl( \kappa- \biggl( C_2+\frac{C_1^2}{1-C_0} \biggr) f_{\mathrm{lo}}^2 \biggr) ^{-1} \\ &\leq C_{\kappa} \lambda_{\mathrm{lo}}^{2}, \end{aligned}$$

for a certain positive constant C _κ . Note that due to the numeric upper bounds on the constants in (B.2) C _κ is indeed a positive constant as long as f _lo≥1. Finally, we obtain a bound on the magnitude of the entries of α

$$\begin{aligned} \| \alpha \| _{\infty} & = \| D_0^{-1}D_1 S^{-1} v \| _{\infty} \leq \| D_0^{-1}D_1 S^{-1} \| _{\infty} \leq \| D_0^{-1} \| _{\infty} \| D_1 \| _{\infty} \| S^{-1} \| _{\infty} \leq C_{\alpha} \lambda_{\mathrm{lo}} , \end{aligned}$$

(B.7)

where C _α =C _κ C ₁/(1−C ₀), and on the entries of β

$$ \| \beta \| _{\infty}=\| S^{-1} v \| _{\infty}\leq \| S^{-1} \| _{\infty} \leq C_{\beta}\lambda_{\mathrm{lo}}^{2}, $$

(B.8)

for a positive constant C _β =C _κ . Combining these inequalities with (B.3) and the fact that the absolute values of G(t) and G ⁽¹⁾(t) are bounded by one and 7f _lo respectively (see the proof of Lemma C.5 in [6]), we have that for any t

$$\begin{aligned} |q_1(t)| & = \biggl \vert \sum_{t_k \in T} \alpha_k G ( t-t_k ) + \sum_{t_k \in T} \beta_k G^{ ( 1 ) } ( t-t_k ) \biggr \vert \\ & \leq\| \alpha \| _{\infty}\sum_{t_k \in T}\bigl \vert G ( t-t_k ) \bigr \vert + \| \beta \| _{\infty}\sum _{t_k \in T} \bigl \vert G^{ ( 1 ) } ( t-t_k ) \bigr \vert \\ & \leq C_{\alpha} \lambda_{\mathrm{lo}} \biggl( \bigl \vert G ( t ) \bigr \vert + \sum_{t_k \in T\setminus\{t_i\}} \bigl \vert G ( t-t_k ) \bigr \vert \biggr) \\ &\quad\ \ {}+ C_{\beta }\lambda_{\mathrm{lo}}^2 \biggl( \bigl \vert G^{ ( 1 ) } ( t ) \bigr \vert + \sum _{t_k \in T\setminus\{t_i\}} \bigl \vert G^{ ( 1 ) } ( t-t_k ) \bigr \vert \biggr) \\ & \leq C \lambda_{\mathrm{lo}}, \end{aligned}$$

(B.9)

where t _i denotes the element in T nearest to t (note that all other elements are at least Δ/2 away). Thus, (2.19) holds.

The proof is completed by the following lemma, which proves (2.18).

Lemma 6.1

For any t _j ∈T and $t \in\mathbb{T}$ obeying |t−t _j |≤0.16λ _lo, we have

$$\begin{aligned} |q_1(t)-v_j ( t-t_j ) | & \leq \frac{C ( t-t_j ) ^2}{\lambda_{{\mathrm{lo}}}}. \end{aligned}$$

Proof

We assume without loss of generality that t _j =0. By symmetry, it suffices to show the claim for t∈(0,0.16λ _lo]. To ease notation, we define w(t)=v _j t−q ₁(t)=w _R (t)+iw _I (t), where w _R is the real part of w and w _I the imaginary part. Leveraging (B.7), (B.8) and (B.2) together with the fact that G ⁽²⁾(t) and G ⁽³⁾(t) are bounded by $4 f_{\mathrm{lo}}^{2}$ and $6 f_{\mathrm{lo}}^{3}$ respectively if |t|≤0.16λ _lo (see the proof of Lemma 2.3 in [6]), we obtain

$$\begin{aligned} \bigl \vert w_R'' ( t ) \bigr \vert & = \biggl \vert \sum_{t_k \in T}\operatorname{Re} ( \alpha_k ) G ^{ ( 2 ) } ( t-t_k ) + \sum _{t_k \in T}\operatorname{Re} ( \beta_k ) G ^{ ( 3 ) } ( t-t_k ) \biggr \vert \\ & \leq\| \alpha \| _{\infty}\sum_{t_k \in T}\bigl \vert G^{ ( 2 ) } ( t-t_k ) \bigr \vert + \| \beta \| _{\infty}\sum_{t_k \in T} \bigl \vert G ^{ ( 3 ) } ( t-t_k ) \bigr \vert \\ & \leq C_{\alpha} \lambda_{\mathrm{lo}} \biggl( \bigl \vert G^{ ( 2 ) } ( t ) \bigr \vert + \sum_{t_k \in T\setminus\{0\} } \bigl \vert G^{ ( 2 ) } ( t-t_k ) \bigr \vert \biggr) \\ &\quad\ \ {} + C_{\beta}\lambda_{\mathrm{lo}}^2 \biggl( \bigl \vert G^{ ( 3 ) } ( t ) \bigr \vert + \sum_{t_k \in T\setminus\{0\}} \bigl \vert G^{ ( 3 ) } ( t-t_k ) \bigr \vert \biggr) \\ & \leq C f_{\mathrm{lo}}. \end{aligned}$$

The same bound applies to w _I . Since w _R (0), $w_{R}'(0)$, w _I (0) and $w_{I}'(0)$ are all equal to zero, this implies |w _R (t)|≤Cf _lo t ²—and similarly for |w _I (t)|—in the interval of interest. Whence, |w(t)|≤Cf _lo t ². □

Appendix C: Proof of Corollary 1.3

The proof of Theorem 1.2 relies on two identities

$$\begin{aligned} \| x_{\mathrm{est}} \| _{\mathrm{TV}} & \leq \| x \| _{\mathrm{TV}}, \end{aligned}$$

(C.1)

$$\begin{aligned} \| Q_{\mathrm{lo}} ( x_{\mathrm{est}}- x ) \| _{L_1} & \leq2\delta, \end{aligned}$$

(C.2)

which suffice to establish

$$ \| K_{\mathrm{hi}}*(x_{\mathrm{est}}- x) \| _{L_1} \leq C_0 \mathrm{SRF}^2 \delta. $$

To prove the corollary, we show that (C.1) and (C.2) hold. Due to the fact that $\| \epsilon \| _{2}^{2}$ follows a χ ²-distribution with 4f _lo+2 degrees of freedom, we have

$$ \mathbb{P} \bigl( \| \epsilon \| _{2} > ( 1+\gamma ) \sigma\sqrt{4 f_{\mathrm{lo}}+2}=\delta \bigr) < e^{- 2f_{\mathrm{lo}}\gamma^2 }, $$

for any positive γ by a concentration inequality (see [24, Sect. 4]). By Parseval, this implies that with high probability $\| Q_{\mathrm{lo}}x -y \| _{\mathcal {L}_{2}}=\| \epsilon \| _{2}\leq\delta$. As a result, x _est is feasible, which implies (C.1) and furthermore

$$\begin{aligned} \| Q_{\mathrm{lo}} ( x_{\mathrm{est}}- x ) \| _{L_1} \leq\| Q_{\mathrm{lo}} ( x_{\mathrm{est}} - x ) \| _{\mathcal{L}_2} \leq \| Q_{\mathrm{lo}}x -y \| _{\mathcal {L}_2}+\| y-Q_{\mathrm{lo}}x_{\mathrm{est}} \| _{\mathcal{L}_2} \leq2\delta, \end{aligned}$$

since by the Cauchy-Schwarz inequality $\| f \| _{L_{1}} \leq\| f \| _{\mathcal{L}_{2}}$ for any function f with bounded L ₂ norm supported on the unit interval. Thus, (C.2) also holds and the proof is complete.

Appendix D: Extension to Multiple Dimensions

The extension of the proof hinges on establishing versions of Lemmas 2.4, 2.5 and 2.7 for multiple dimensions. These lemmas construct bounded low-frequency polynomials which interpolate a sign pattern on a well-separated set of points S and have bounded second derivatives in a neighborhood of S. In the multidimensional case, we need the directional derivative of the polynomials to be bounded in any direction, which can be ensured by bounding the eigenvalues of their Hessian matrix evaluated on the support of the signal. To construct such polynomials one can proceed in a way similar to the proof of Lemmas 2.4 and 2.7, namely, by using a low-frequency kernel constructed by tensorizing several squared Fejér kernels to interpolate the sign pattern, while constraining the first-order derivatives to either vanish or have a fixed value. As in the one-dimensional case, one can set up a system of equations and prove that it is well conditioned using the rapid decay of the interpolation kernel away from the origin. Finally, one can verify that the construction satisfies the required conditions by exploiting the fact that the interpolation kernel and its derivatives are locally quadratic and rapidly decaying. This is spelled out in the proof of Proposition C.1 in [6] to prove a version of Lemma 2.4 in two dimensions. In order to clarify further how to adapt our techniques to a multidimensional setting we provide below a sketch of the proof of the analog of Lemma 2.1 in two dimensions. In particular, this illustrates how the increase in dimension does not change the exponent of the SRF in our recovery guarantees.

4.1 D.1 Proof of an Extension of Lemma 2.1 to Two Dimensions

We now have $t \in\mathbb{T}^{2}$. As a result, we redefine

$$\begin{aligned} S_{\mathrm{near}}^{\lambda} ( j ) & := \{ t : \| t-t_j \| _{L_\infty}\leq w \lambda \}, \\ S_{\mathrm{far}}^{\lambda} &:= \{ t : \| t-t_j \| _{L_\infty}> w \lambda, \forall t_j \in T \}, \\ I_{S_{\mathrm{near}}^{\lambda} ( j ) } ( \mu ) &:= \frac{1}{\lambda_{\mathrm{lo}}^2}\int_{S_{\mathrm{near}}^{\lambda } ( j ) } \| t-t_j \| _{2}^2 \vert \mu \vert ( d t ) , \end{aligned}$$

where w is a constant.

The proof relies on the existence of a low-frequency polynomial

$$ q(t) = \sum_{k_1 = -f_{{\mathrm{lo}}}}^{f_{{\mathrm {lo}}}}\sum _{k_2 = -f_{{\mathrm{lo}}}}^{f_{{\mathrm{lo}}}} c_{k_1,k_2} e^{i2\pi(k_1 t_1 +k_2 t_2)} $$

satisfying

$$\begin{aligned} q(t_j) & = v_j, \quad t_j \in T, \end{aligned}$$

(D.1)

$$\begin{aligned} |q(t)| & \leq1-\frac{C'_a \| t-t_j \| _{2}^2}{\lambda_{{\mathrm{lo}}}^2} , \quad t \in S_{\mathrm{near}}^{\lambda_{{\mathrm{lo}}}} ( j ) , \end{aligned}$$

(D.2)

$$\begin{aligned} |q(t)| & < 1-C'_b , \quad t \in S_{\mathrm{far}}^{\lambda_{{\mathrm{lo}}}}, \end{aligned}$$

(D.3)

where $C_{a}'$ and $C_{b}'$ are constants. Proposition C.1 in [6] constructs such a polynomial. Under a minimum distance condition, which constrains the elements of T to be separated by 2.38λ _lo in infinity norm (as explained in [6] this choice of norm is arbitrary and could be changed to the ℓ ₂ norm), [6] shows that q satisfies (D.1) and (D.3) and that both eigenvalues of its Hessian matrix evaluated on T are of order $f_{\mathrm{lo}}^{2}$, which implies (D.2).

As in one dimension, we perform a polar decomposition of P _T h,

$$\begin{aligned} P_{T} h = e^{i \phi ( t ) }\vert P_{T} h \vert , \end{aligned}$$

and work with $v_{j} = e^{-i \phi(t_{j})}$. The rest of the proof is almost identical to the 1D case. Since q is low frequency,

$$\begin{aligned} \biggl \vert \int_{\mathbb{T}^2} q(t) dh (t) \biggr \vert & \leq2 \delta. \end{aligned}$$

(D.4)

Next, since q interpolates e ^−iϕ(t) on T,

$$\begin{aligned} \| P_{T} h \| _{\mathrm{TV}} = \int_{\mathbb {T}^2} q(t) P_{T} h ( dt ) &\leq2\delta+ \sum_{j\in T} \biggl \vert \int_{S_{\mathrm{near}}^{\lambda_{\mathrm{lo}}} ( j ) \setminus \{ t_j \} } q(t) h ( dt ) \biggr \vert + \biggl \vert \int_{S_{\mathrm{far}}^{\lambda_{\mathrm {lo}}}} q(t) h ( dt ) \biggr \vert . \end{aligned}$$

(D.5)

Applying (D.3) and Hölder’s inequality, we obtain

$$\begin{aligned} \biggl \vert \int_{S_{\mathrm{far}}^{\lambda_{\mathrm{lo}}}} q(t) h ( dt ) \biggr \vert & \leq \bigl( 1-C_b' \bigr) \| P_{S_{\mathrm{far}}^{\lambda_{\mathrm{lo}}}} ( h ) \| _{\mathrm{TV}}. \end{aligned}$$

(D.6)

Setting t _j =(0,0) without loss of generality, the triangle inequality and (D.2) yield

$$\begin{aligned} \biggl \vert \int_{S_{\mathrm{near}}^{\lambda_{\mathrm{lo}}} ( j ) \setminus \{ (0,0) \}} q(t) h ( dt ) \biggr \vert & \leq \int_{S_{\mathrm{near}}^{\lambda_{\mathrm{lo}}} ( j ) \setminus \{ (0,0) \}} \vert h \vert ( dt ) - C_a' I_{S_{\mathrm{near}}^{\lambda_{\mathrm {lo}}} ( j ) } ( h ) . \end{aligned}$$

(D.7)

Combining (D.5), (D.6) and (D.7) gives

$$\begin{aligned} \| P_{T} h \| _{\mathrm{TV}} \leq& 2\delta+ \| P_{T^{c}}h \| _{\mathrm{TV}} - C_b' \| P_{S_{\mathrm{far}}^{\lambda_{\mathrm{lo}}}} ( h ) \| _{\mathrm{TV}}- C_a' I_{S_{\mathrm {near}}^{\lambda_{\mathrm{lo}}}} ( h ) \end{aligned}$$

and similarly

$$\| P_{T} h \| _{\mathrm{TV}} \leq2\delta+ \| P_{T^{c}}h \| _{\mathrm{TV}} - w^2 C_a' \mathrm{SRF}^{-2} \| P_{S_{\mathrm{far}}^{\lambda _{\mathrm{hi}}}} ( h ) \| _{\mathrm{TV}}- C_a' I_{S_{\mathrm{near}} ^{\lambda_{\mathrm{hi}}}} ( h ) . $$

By the same argument as in the 1D case, the fact that $\hat{x}$ has minimal total-variation norm is now sufficient to establish

$$\begin{aligned} C_b' \| P_{S_{\mathrm{far}}^{\lambda_{\mathrm {lo}}}} ( h ) \| _{\mathrm{TV}} + C_a' I_{S_{\mathrm{near}}^{\lambda_{\mathrm{lo}} }} ( h ) \leq2 \delta, \end{aligned}$$

and

$$\begin{aligned} w^2 C_a' \mathrm{SRF}^{-2} \| P_{S_{\mathrm {far}}^{\lambda_{\mathrm{hi}}}} ( h ) \| _{\mathrm{TV}} + C_a' I_{S_{\mathrm{near}}^{\lambda_{\mathrm{hi}}}} ( h ) \leq2 \delta. \end{aligned}$$