A New Class of Fully Discrete Sparse Fourier Transforms: Faster Stable Implementations with Guarantees

Abstract

In this paper we consider sparse Fourier transform (SFT) algorithms for approximately computing the best s-term approximation of the discrete Fourier transform (DFT) ${\hat{\mathbf {f}}}\in {{\mathbb {C}}}^N$ of any given input vector $\mathbf {f}\in {{\mathbb {C}}}^N$ in just $\left( s \log N\right) ^{{{\mathcal {O}}}(1)}$-time using only a similarly small number of entries of $\mathbf {f}$. In particular, we present a deterministic SFT algorithm which is guaranteed to always recover a near best s-term approximation of the DFT of any given input vector $\mathbf {f}\in {{\mathbb {C}}}^N$ in ${{\mathcal {O}}} \left( s^2 \log ^{\frac{11}{2}} (N) \right)$-time. Unlike previous deterministic results of this kind, our deterministic result holds for both arbitrary vectors $\mathbf {f}\in {{\mathbb {C}}}^N$ and vector lengths N. In addition to these deterministic SFT results, we also develop several new publicly available randomized SFT implementations for approximately computing ${\hat{\mathbf {f}}}$ from $\mathbf {f}$ using the same general techniques. The best of these new implementations is shown to outperform existing discrete sparse Fourier transform methods with respect to both runtime and noise robustness for large vector lengths N.

Appendices

Appendix A: Fourier Basics: Continuous versus Discrete Fourier Transforms for Trigonometric Polynomials

Our objective in this appendix is to provide additional details regarding (2) and its relationship to the continuous sparse Fourier transform methods for periodic functions that we employ herein. Our starting point will be to assume only that we have been provided with a vector of data $\mathbf {f}\in {{\mathbb {C}}}^{N}$. Our goal is to rapidly approximate the matrix vector product ${\hat{\mathbf {f}}}= F \mathbf {f}$ where $F\in {{\mathbb {C}}}^{N\times N}$ is the DFT matrix whose entries are given by

$$\begin{aligned} F_{\omega ,j} :=\frac{(-1)^\omega }{N} \mathbb {e}^{-\frac{2\pi \mathbb {i}\cdot \omega \cdot j}{N}} \end{aligned}$$

for $\omega \in \left( -\left\lceil \frac{N}{2}\right\rceil ,\left\lfloor \frac{N}{2}\right\rfloor \right] \cap {\mathbb {Z}}$ and $j = 0, \dots , N-1$.

Beginning from this starting point, one may choose to regard the given vector of data $\mathbf {f}$ as having been generated by sampling a $2 \pi$-periodic trigonometric polynomial $f: [-\pi , \pi ] \rightarrow {{\mathbb {C}}}$ of the form

$$\begin{aligned} f\left( x\right) =\sum _{\omega \in B}\widehat{f}_{\omega } \mathbb {e}^{\mathbb {i}\omega x} \end{aligned}$$

where $B:=\left( -\left\lceil \frac{N}{2}\right\rceil ,\left\lfloor \frac{N}{2}\right\rfloor \right] \cap {\mathbb {Z}}$. In particular, herein we will assume that $\mathbf {f}$ has its jth entry generated by $f_j := f\left( -\pi +\frac{2\pi j}{N}\right)$ for $j = 0, \dots , N-1$. Note that there is exactly one such f for the given data $\mathbf {f}$ since $|B| = N$ (i.e., f is the unique interpolating polynomial for $\mathbf {f}$ with $\omega \in B$).

Considering f just above, we can now see that for any $k \in \mathbb {Z}$ the associated Fourier series coefficient of f is

$$\begin{aligned} \frac{1}{2\pi }\int _{-\pi }^{\pi }f\left( x\right) \mathbb {e}^{-\mathbb {i}k x}~dx&~=~ \frac{1}{2\pi }\int _{-\pi }^{\pi } \left( \sum _{\omega \in B} \widehat{f}_{\omega } \mathbb {e}^{\mathbb {i}\omega x} \right) \mathbb {e}^{-\mathbb {i}k x}~dx \\&~=~ \sum _{\omega \in B} \widehat{f}_{\omega } \left( \frac{1}{2\pi }\int _{-\pi }^{\pi } \mathbb {e}^{\mathbb {i}(\omega - k) x}~dx \right) \\&~=~ {\left\{ \begin{array}{ll} \widehat{f}_{k}, &{}\quad k \in B \\ 0, &{}\quad \text {otherwise} \end{array}\right. }. \end{aligned}$$

Changing our focus now to the discrete Fourier transform of $\mathbf {f}$ considered as being samples from f we can see that

Note that the line above establishes (2) where the vector ${\hat{\mathbf {f}}}\in {{\mathbb {C}}}^N$ exactly contains the nonzero Fourier series coefficients of f as its entries. As a result we can see that computing the Fourier series coefficients of f is equivalent to computing the matrix vector product ${\hat{\mathbf {f}}}= F \mathbf {f}$ for our given data $\mathbf {f}$.

Appendix B: Proof of Lemmas 1, 2 and 3

We will restate each lemma before its proof for ease of reference.

Lemma 14

(Restatement of Lemma 1) The $2\pi \hbox {-periodic}$ Gaussian $g:\left[ -\pi ,\pi \right] \rightarrow {\mathbb {R}}^{+}$ has

$$\begin{aligned} g\left( x\right) \le \left( \frac{3}{c_1}+\frac{1}{\sqrt{2\pi }} \right) \mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}} \end{aligned}$$

for all $x \in \left[ -\pi ,\pi \right]$.

Proof

Observe that

$$\begin{aligned} c_{1}g\left( x\right) =\sum _{n=-\infty }^{\infty }\mathbb {e}^{-\frac{\left( x-2n\pi \right) ^{2}}{2c_{1}^{2}}}&=\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\mathbb {e}^{-\frac{\left( x-2\pi \right) ^{2}}{2c_{1}^{2}}}+\mathbb {e}^{-\frac{\left( x+2\pi \right) ^{2}}{2c_{1}^{2}}}+\sum _{\left| n\right| \ge 2}\mathbb {e}^{-\frac{\left( x-2n\pi \right) ^{2}}{2c_{1}^{2}}}\\&\le 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\int _{1}^{\infty }\mathbb {e}^{-\frac{\left( x-2n\pi \right) ^{2}}{2c_{1}^{2}}}dn+\int _{1}^{\infty }\mathbb {e}^{-\frac{\left( x+2n\pi \right) ^{2}}{2c_{1}^{2}}}dn \end{aligned}$$

holds since the series above have monotonically decreasing positive terms, and $x\in \left[ -\pi ,\pi \right]$.

Now, if $x\in \left[ 0,\pi \right]$ and $n\ge 1$, one has

$$\begin{aligned} \mathbb {e}^{-\frac{\left( 2n+1\right) ^{2}\pi ^{2}}{2c_{1}^{2}}}\le \mathbb {e}^{-\frac{\left( x+2n\pi \right) ^{2}}{2c_{1}^{2}}}\le \mathbb {e}^{-\frac{4n^{2}\pi ^{2}}{2c_{1}^{2}}}\le \mathbb {e}^{-\frac{\left( x-2n\pi \right) ^{2}}{2c_{1}^{2}}}\le \mathbb {e}^{-\frac{\left( 2n-1\right) ^{2}\pi ^{2}}{2c_{1}^{2}}}, \end{aligned}$$

which yields

$$\begin{aligned} c_{1}g\left( x\right)&~\le ~3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+2\int _{1}^{\infty }\mathbb {e}^{-\frac{\pi ^{2}\left( 2n-1\right) ^{2}}{2c_{1}^{2}}}dn \\&~=~ 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{1}{2}\left( \int _{-\infty }^{\infty }\mathbb {e}^{-\frac{\pi ^{2}m^{2}}{2c_{1}^{2}}}dm-\int _{-1}^{1}\mathbb {e}^{-\frac{\pi ^{2}m^{2}}{2c_{1}^{2}}}dm\right) \\&=3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{c_{1}}{\sqrt{2\pi }}-\frac{1}{2}\int _{-1}^{1}\mathbb {e}^{-\frac{\pi ^{2}m^{2}}{2c_{1}^{2}}}dm. \end{aligned}$$

Using Lemma 8 to bound the last integral we can now get that

$$\begin{aligned} c_{1}g\left( x\right)&\le 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{c_{1}}{\sqrt{2\pi }}-\frac{1}{2}\frac{\sqrt{2}c_{1}}{\pi }\sqrt{\pi \left( 1-\mathbb {e}^{-\frac{\pi ^{2}}{2c_{1}^{2}}}\right) }\\&= 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{c_{1}}{\sqrt{2\pi }}\left( 1-\sqrt{\left( 1-\mathbb {e}^{-\frac{\pi ^{2}}{2c_{1}^{2}}}\right) }\right) \\&\le 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{c_{1}}{\sqrt{2\pi }}\mathbb {e}^{-\frac{\pi ^{2}}{2c_{1}^{2}}} \le 3\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}+\frac{c_{1}}{\sqrt{2\pi }}\mathbb {e}^{-\frac{x^{2}}{2c_{1}^{2}}}. \end{aligned}$$

Recalling now that g is even we can see that this inequality will also hold for all $x \in [-\pi ,0]$ as well. $\square$

Lemma 15

(Restatement of Lemma 2) The $2\pi \hbox {-periodic}$ Gaussian $g:\left[ -\pi ,\pi \right] \rightarrow {\mathbb {R}}^{+}$ has

$$\begin{aligned} \widehat{g}_{\omega } = \frac{1}{\sqrt{2\pi }}\mathbb {e}^{-\frac{c_1^2 \omega ^2}{2}} \end{aligned}$$

for all $\omega \in {\mathbb {Z}}$. Thus, $\widehat{g}=\left\{ \widehat{g}_{\omega }\right\} _{\omega \in {\mathbb {Z}}}\in \ell ^{2}$ decreases monotonically as $|\omega |$ increases, and also has $\Vert \widehat{g} \Vert _{\infty } = \frac{1}{\sqrt{2 \pi }}$.

Proof

Starting with the definition of the Fourier transform, we calculate

$$\begin{aligned} \widehat{g}_{\omega }= & {} \frac{1}{c_1}\sum _{n=-\infty }^{\infty }\frac{1}{2\pi }\int _{-\pi }^{\pi }\mathbb {e}^{-\frac{(x-2n\pi )^{2}}{2c_1^{2}}}\mathbb {e}^{-\mathbb {i}\omega x}~dx\\= & {} \frac{1}{c_1}\sum _{n=-\infty }^{\infty }\frac{1}{2\pi }\int _{-\pi }^{\pi }\mathbb {e}^{-\frac{(x-2n\pi )^{2}}{2c_1^{2}}}\mathbb {e}^{-\mathbb {i}\omega (x-2n\pi )}~dx\\= & {} \frac{1}{c_1}\sum _{n=-\infty }^{\infty }\frac{1}{2\pi }\int _{-\pi -2n\pi }^{\pi -2n\pi }\mathbb {e}^{-\frac{u{}^{2}}{2c_1^{2}}}\mathbb {e}^{-\mathbb {i}\omega u}~du\\= & {} \frac{1}{2\pi c_1}\int _{-\infty }^{\infty }\mathbb {e}^{-\frac{u{}^{2}}{2c_1^{2}}} \mathbb {e}^{-\mathbb {i}\omega u}~du\\= & {} \frac{c_1\sqrt{2\pi }}{2\pi c_1} \mathbb {e}^{-\frac{c_1^{2}\omega ^{2}}{2}}\\= & {} \frac{\mathbb {e}^{-\frac{c_1^{2}\omega ^{2}}{2}}}{\sqrt{2\pi }}. \end{aligned}$$

The last two assertions now follow easily. $\square$

Lemma 16

(Restatement of Lemma 3) Choose any $\tau \in \left( 0, \frac{1}{\sqrt{2\pi }} \right)$, $\alpha \in \left[ 1, \frac{N}{\sqrt{\ln N}} \right]$, and $\beta \in \left( 0 , \alpha \sqrt{\frac{\ln \left( 1/\tau \sqrt{2\pi } \right) }{2}} ~\right]$. Let $c_1 = \frac{\beta \sqrt{\ln N}}{N}$ in the definition of the periodic Gaussian g from (3). Then $\widehat{g}_{\omega } \in \left[ \tau , \frac{1}{\sqrt{2\pi }} \right]$ for all $\omega \in {\mathbb {Z}}$ with $|\omega | \le \Bigl \lceil \frac{N}{\alpha \sqrt{\ln N}}\Bigr \rceil$.

Proof

By Lemma 2 above it suffices to show that

$$\begin{aligned} \frac{1}{\sqrt{2\pi }}\mathbb {e}^{-\frac{c_1^2 \left( \bigl \lceil \frac{N}{\alpha \sqrt{\ln N}}\bigr \rceil \right) ^2}{2}} \ge \tau , \end{aligned}$$

which holds if and only if

$$\begin{aligned} c_1^2 \left( \left\lceil \frac{N}{\alpha \sqrt{\ln N}}\right\rceil \right) ^2\le & {} 2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) \\ c_1\le & {} \frac{\sqrt{2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) }}{\left\lceil \frac{N}{\alpha \sqrt{\ln N}}\right\rceil }. \end{aligned}$$

Thus, it is enough to have

$$\begin{aligned} c_1 \le \frac{\sqrt{2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) }}{ \frac{N}{\alpha \sqrt{\ln N}} + 1} ~=~\frac{\alpha \sqrt{2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) \ln N}}{ N + \alpha \sqrt{\ln N}}, \end{aligned}$$

or,

$$\begin{aligned} c_1 = \frac{\beta \sqrt{\ln N}}{N} \le \frac{\alpha \sqrt{2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) \ln N}}{2N} \le \frac{\alpha \sqrt{2 \ln \left( \frac{1}{\tau \sqrt{2\pi }}\right) \ln N}}{ N + \alpha \sqrt{\ln N}}. \end{aligned}$$

This, in turn, is guaranteed by our choice of $\beta$. $\square$

Appendix C: Proof of Lemma 4 and Theorem 2

We will restate Lemma 4 before its proof for ease of reference.

Lemma 17

(Restatement of Lemma 4) Let $s, \epsilon ^{-1} \in {\mathbb {N}} \setminus \{ 1 \}$ with $(s/\epsilon ) \ge 2$, and $\mathbf {n}\in {\mathbb {C}}^m$ be an arbitrary noise vector. There exists a set of m points $\left\{ x_k \right\} ^m_{k=1} \subset [-\pi , \pi ]$ such that Algorithm 3 on page 72 of [21], when given access to the corrupted samples $\left\{ f(x_k) + n_k \right\} ^m_{k=1}$, will identify a subset $S \subseteq B$ which is guaranteed to contain all $\omega \in B$ with

$$\begin{aligned} \left| \widehat{f}_{\omega } \right| > 4 \left( \frac{\epsilon \cdot \left\| {\hat{\mathbf {f}}}- {\hat{\mathbf {f}}}^{\mathrm{opt}}_{(s/\epsilon )} \right\| _1}{s} + \left\| \widehat{f} - \widehat{f}\vert _{B} \right\| _1 + \Vert \mathbf {n}\Vert _\infty \right) . \end{aligned}$$

(15)

Furthermore, every $\omega \in S$ returned by Algorithm 3 will also have an associate Fourier series coefficient estimate $z_{\omega } \in {\mathbb {C}}$ which is guaranteed to have

$$\begin{aligned} \left| \widehat{f}_{\omega } - z_{\omega } \right| \le \sqrt{2} \left( \frac{\epsilon \cdot \left\| {\hat{\mathbf {f}}}- {\hat{\mathbf {f}}}^{\mathrm{opt}}_{(s/\epsilon )} \right\| _1}{s} + \left\| \widehat{f} - \widehat{f}\vert _{B} \right\| _1 + \Vert \mathbf {n}\Vert _\infty \right) . \end{aligned}$$

(16)

Both the number of required samples, m, and Algorithm 3’s operation count are

$$\begin{aligned} {\mathcal {O}} \left( \frac{s^2 \cdot \log ^4 (N)}{\log \left( \frac{s}{\epsilon } \right) \cdot \epsilon ^2} \right) . \end{aligned}$$

(17)

If succeeding with probability $(1-\delta ) \in [2/3,1)$ is sufficient, and $(s/\epsilon ) \ge 2$, the Monte Carlo variant of Algorithm 3 referred to by Corollary 4 on page 74 of [21] may be used. This Monte Carlo variant reads only a randomly chosen subset of the noisy samples utilized by the deterministic algorithm,

$$\begin{aligned} \left\{ f(\tilde{x}_k) + \tilde{n}_k \right\} ^{\tilde{m}}_{k=1} \subseteq \left\{ f(x_k) + n_k \right\} ^m_{k=1}, \end{aligned}$$

yet it still outputs a subset $S \subseteq B$ which is guaranteed to simultaneously satisfy both of the following properties with probability at least $1-\delta$:

• (i) S will contain all $\omega \in B$ satisfying (15), and

• (ii) all $\omega \in S$ will have an associated coefficient estimate $z_{\omega } \in {\mathbb {C}}$ satisfying (16).

Finally, both this Monte Carlo variant’s number of required samples, $\tilde{m}$, as well as its operation count will also always be

$$\begin{aligned} {\mathcal {O}} \left( \frac{s}{\epsilon } \cdot \log ^3 (N) \cdot \log \left( \frac{N}{\delta } \right) \right) . \end{aligned}$$

(18)

Proof

The proof of this lemma involves a somewhat tedious and uninspired series of minor modifications to various results from [21]. In what follows we will outline the portions of that paper which need to be changed in order to obtain the stated lemma. Algorithm 3 on page 72 of [21] will provide the basis of our discussion.

In the first paragraph of our lemma we are provided with m-contaminated evaluations of f, $\left\{ f(x_k) + n_k \right\} ^m_{k=1}$, at the set of m points $\left\{ x_k \right\} ^m_{k=1} \subset [-\pi , \pi ]$ required by line 4 of Algorithm 1 on page 67 of [21]. These contaminated evaluations of f will then be used to approximate the vector $\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A} \in {\mathbb {C}}^m$ in line 4 of Algorithm 3. More specifically, using (18) on page 67 of [21] one can see that each $\left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j \in {\mathbb {C}}$ is effectively computed via a DFT

$$\begin{aligned} \left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j = \frac{1}{s_j} \sum ^{s_j - 1}_{k=0} f \left( -\pi + \frac{2 \pi k}{s_j} \right) \mathbb {e}^{\frac{-2 \pi \mathbb {i}k h_j}{s_j}} \end{aligned}$$

(19)

for some integers $0 \le h_j < s_j$. Note that we are guaranteed to have noisy evaluations of f at each of these points by assumption. That is, we have $f \left( x_{j,k} \right) + n_{j,k}$ for all $x_{j,k} := -\pi + \frac{2 \pi k}{s_j}$, $k = 0, \dots , s_j - 1$.

We therefore approximate each $\left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j$ via an approximate DFT as per (19) by

$$\begin{aligned} E_j := \frac{1}{s_j} \sum ^{s_j - 1}_{k=0} \left( f \left( x_{j,k} \right) + n_{j,k} \right) \mathbb {e}^{\frac{-2 \pi \mathbb {i}k h_j}{s_j}}. \end{aligned}$$

One can now see that

$$\begin{aligned} \left| E_j - \left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j \right| = \left| \frac{1}{s_j} \sum ^{s_j - 1}_{k=0} n_{j,k} \mathbb {e}^{\frac{-2 \pi \mathbb {i}k h_j}{s_j}} \right| \le \frac{1}{s_j} \sum ^{s_j - 1}_{k=0} \left| n_{j,k} \right| \le \Vert \mathbf {n}\Vert _{\infty } \end{aligned}$$

(20)

holds for all j. Every entry of both ${\mathcal {E}_{s_1,K} \tilde{\psi } \mathbf{A}}$ and ${\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}}$ referred to in Algorithm 3 will therefore be effectively replaced by its corresponding $E_j$ estimate. Thus, the lemma we seek to prove is essentially obtained by simply incorporating the additional error estimate (20) into the analysis of Algorithm 3 in [21] wherever an ${\mathcal {E}_{s_1,K} \tilde{\psi } \mathbf{A}}$ or ${\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}}$ currently appears.

To show that lines 6 – 14 of Algorithm 3 will identify all $\omega \in B$ satisfying (15) we can adapt the proof of Lemma 6 on page 72 of [21]. Choose any $\omega \in B$ you like. Lemmas 3 and 5 from [21] together with (20) above ensure that both

$$\begin{aligned} \left| E_j - \widehat{f}_{\omega } \right|&\le \left| E_j - \left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j \right| + \left| \left( {\mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j - \widehat{f}_{\omega } \right| \nonumber \\&\le \Vert \mathbf {n}\Vert _{\infty } + \frac{\epsilon \cdot \left\| {\hat{\mathbf {f}}}- {\hat{\mathbf {f}}}^{\mathrm{opt}}_{(s/\epsilon )} \right\| _1}{s} + \left\| \widehat{f} - \widehat{f}\vert _{B} \right\| _1 \end{aligned}$$

(21)

and

$$\begin{aligned} \left| E_{j'} - \widehat{f}_{\omega } \right|&\le \left| E_{j'} - \left( {\mathcal {E}_{s_1,K} \tilde{\psi } \mathbf{A}} \right) _{j'} \right| + \left| \left( {\mathcal {E}_{s_1,K} \tilde{\psi } \mathbf{A}} \right) _{j'} - \widehat{f}_{\omega } \right| \nonumber \\&\le \Vert \mathbf {n}\Vert _{\infty } + \frac{\epsilon \cdot \left\| {\hat{\mathbf {f}}}- {\hat{\mathbf {f}}}^{\mathrm{opt}}_{(s/\epsilon )} \right\| _1}{s} + \left\| \widehat{f} - \widehat{f}\vert _{B} \right\| _1 \end{aligned}$$

(22)

hold for more than half of the j and $j'$-indexes that Algorithm 3 uses to approximate $\widehat{f}_{\omega }$. The rest of the proof of Lemma 6 now follows exactly as in [21] after the $\delta$ at the top of page 73 is redefined to be $\delta := \frac{\epsilon \cdot \left\| {\hat{\mathbf {f}}}- {\hat{\mathbf {f}}}^{\mathrm{opt}}_{(s/\epsilon )} \right\| _1}{s} + \left\| \widehat{f} - \widehat{f}\vert _{B} \right\| _1 + \Vert \mathbf {n}\Vert _\infty$, each $\left( { \mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j$ entry is replaced by $E_j$, and each $\left( {\mathcal {E}_{s_1,K} \tilde{\psi } \mathbf{A}} \right) _{j'}$ entry is replaced by $E_{j'}$.

Similarly, to show that lines 15 – 18 of Algorithm 3 will produce an estimate $z_{\omega } \in {\mathbb {C}}$ satisfying (16) for every $\omega \in S$ one can simply modify the first few lines of the proof of Theorem 7 in Appendix F of [21]. In particular, one can redefine $\delta$ as above, replace the appearance of each $\left( { \mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A}} \right) _j$ entry by $E_j$, and then use (21). The bounds on the runtime follow from the last paragraph of the proof of Theorem 7 in Appendix F of [21] with no required changes. To finish, we note that the second paragraph of the lemma above follows from a completely analogous modification of the proof of Corollary 4 in Appendix G of [21]. $\square$

1.1 Appendix C.1: Proof of Theorem 2

To get the first paragraph of Theorem 2 one can simply utilize the proof of Theorem 7 exactly as it is written in Appendix F of [21] after redefining $\delta$ as above, and then replacing the appearance of each $\left( \mathcal {G}_{\lambda ,K} \tilde{\psi } \mathbf{A} \right) _j$ entry with its approximation $E_j$. Once this has been done, equation (42) in the proof of Theorem 7 can then be taken as a consequence of Lemma 4 above. In addition, all references to Lemma 6 of [21] in the proof can then also be replaced with appeals to Lemma 4 above. To finish, the proof of Corollary 4 in Appendix G of [21] can now be modified in a completely analogous fashion in order to prove the second paragraph of Theorem 2.