Infinite-Dimensional $\ell ^1$ Minimization and Function Approximation from Pointwise Data

Abstract

We consider the problem of approximating a smooth function from finitely many pointwise samples using $\ell ^1$ minimization techniques. In the first part of this paper, we introduce an infinite-dimensional approach to this problem. Three advantages of this approach are as follows. First, it provides interpolatory approximations in the absence of noise. Second, it does not require a priori bounds on the expansion tail in order to be implemented. In particular, the truncation strategy we introduce as part of this framework is independent of the function being approximated, provided the function has sufficient regularity. Third, it allows one to explain the key role weights play in the minimization, namely, that of regularizing the problem and removing aliasing phenomena. In the second part of this paper, we present a worst-case error analysis for this approach. We provide a general recipe for analyzing this technique for arbitrary deterministic sets of points. Finally, we use this tool to show that weighted $\ell ^1$ minimization with Jacobi polynomials leads to an optimal method for approximating smooth, one-dimensional functions from scattered data.

Appendix: Jacobi Polynomials

Given $\alpha ,\beta > -1$, let $P^{(\alpha ,\beta )}_j$ be the Jacobi polynomial of degree j. These polynomials are orthogonal on $D=(-1,1)$ with respect to $\nu ^{(\alpha ,\beta )}(t) = (1-t)^\alpha (1+t)^{\beta }$, with

$$\begin{aligned} \langle P^{(\alpha ,\beta )}_j, P^{(\alpha ,\beta )}_k \rangle _{L^2_{\nu ^{(\alpha ,\beta )}}} = \delta _{j,k} \kappa ^{(\alpha ,\beta )}_j, \end{aligned}$$

where

$$\begin{aligned} \kappa ^{(\alpha ,\beta )}_j = \frac{2^{\alpha +\beta +1}}{2j+\alpha +\beta +1} \frac{\Gamma (j+\alpha +1) \Gamma (j+\beta +1)}{j! \Gamma (j+\alpha +\beta +1)}, \end{aligned}$$

(A.1)

and have the normalization

$$\begin{aligned} P^{(\alpha ,\beta )}_{j}(1) = \left( \begin{array}{c} j + \alpha \\ j \end{array} \right) . \end{aligned}$$

The corresponding orthonormal polynomials are defined by $\phi _j(t) = \left( \kappa ^{(\alpha ,\beta )}_{j-1} \right) ^{-1/2} P^{(\alpha ,\beta )}_{j-1}(t)$, $j \in \mathbb {N}$. Note that

$$\begin{aligned} \kappa ^{(\alpha ,\beta )}_{j} \sim 2^{\alpha +\beta } j^{-1} ,\quad j \rightarrow \infty , \end{aligned}$$

(A.2)

and also that

$$\begin{aligned} P^{(\alpha ,\beta )}_{j}(1) \sim \frac{j^{\alpha }}{\Gamma (\alpha +1)}. \end{aligned}$$

The polynomials $P^{(\alpha ,\beta )}_j$ satisfy the differential equation

$$\begin{aligned} -\left( \nu ^{(\alpha +1,\beta +1)} \left( P^{(\alpha ,\beta )}_{j} \right) ' \right) ' + \lambda ^{(\alpha ,\beta )}_j \nu ^{(\alpha ,\beta )} P^{(\alpha ,\beta )}_j = 0, \end{aligned}$$

(A.3)

where $\lambda ^{(\alpha ,\beta )}_{j} = j(j+\alpha +\beta +1)$. In particular, the derivatives $(P^{(\alpha ,\beta )}_{j})'$ are orthogonal with respect to $\nu ^{(\alpha +1,\beta +1)}$ and satisfy

$$\begin{aligned} \left( P^{(\alpha ,\beta )}_{j} \right) ' = \sqrt{\frac{\lambda ^{(\alpha ,\beta )}_j \kappa ^{(\alpha ,\beta )}_{j}}{\kappa ^{(\alpha +1,\beta +1)}_{j-1}} } P^{(\alpha +1,\beta +1)}_{j-1}. \end{aligned}$$

(A.4)

Lemma A.1

Let $\alpha ,\beta > -1$. Then $\Vert p' \Vert _{L^2_{\nu ^{(\alpha +1,\beta +1)}}} \le \lambda ^{(\alpha ,\beta )}_M \Vert p \Vert _{L^2_{\nu ^{(\alpha ,\beta )}}}$, $\forall p \in \mathbb {P}_M$.

Proof

Let $p \in \mathbb {P}_M$ be arbitrary, and observe that

$$\begin{aligned} p (t)= \sum ^{M}_{j=0} \frac{x_j}{\kappa ^{(\alpha ,\beta )}_j} P^{(\alpha ,\beta )}_j(t),\qquad x_j = \langle p, P^{(\alpha ,\beta )}_j \rangle _{L^2_{\nu ^{(\alpha ,\beta )}}}. \end{aligned}$$

Note that

$$\begin{aligned} \Vert p \Vert ^2_{L^2_{\nu ^{(\alpha ,\beta )}}} = \sum ^{M}_{j=0} \frac{|x_j |^2}{\kappa ^{(\alpha ,\beta )}_j}. \end{aligned}$$

(A.5)

Similarly,

$$\begin{aligned} p'(t) = \sum ^{M-1}_{j=0} \frac{y_j}{\kappa ^{(\alpha +1,\beta +1)}_{j}} P^{(\alpha +1,\beta +1)}_{j}(t),\qquad y_j = \langle p', P^{(\alpha +1,\beta +1)}_j \rangle _{L^2_{\nu ^{(\alpha +1,\beta +1)}}}, \end{aligned}$$

and

$$\begin{aligned} \Vert p' \Vert ^2_{L^2_{\nu ^{(\alpha +1,\beta +1)}}} = \sum ^{M-1}_{j=0} \frac{|y_j |^2}{\kappa ^{(\alpha +1,\beta +1)}_j}. \end{aligned}$$

(A.6)

Consider $x_j$. By the differential equation (A.3) and the fact that $\nu ^{(\alpha +1,\beta +1)}(\pm 1 ) = 0$, we have

$$\begin{aligned} x_j = \int ^{1}_{-1} p(t) P^{(\alpha ,\beta )}_j(t) \nu ^{(\alpha ,\beta )}(t) \,\mathrm {d}t = \frac{1}{\lambda ^{(\alpha ,\beta )}_j} \int ^{1}_{-1} p'(t) \left( P^{(\alpha ,\beta )}_{j}(t) \right) ' \nu ^{(\alpha +1,\beta +1)}(t) \,\mathrm {d}t. \end{aligned}$$

Hence, by (A.4),

$$\begin{aligned} x_j&= \sqrt{\frac{\kappa ^{(\alpha ,\beta )}_{j}}{\lambda ^{(\alpha ,\beta )}_j \kappa ^{(\alpha +1,\beta +1)}_{j-1}} } \langle p', P^{(\alpha +1,\beta +1)}_{j-1} \rangle _{L^2_{\nu ^{(\alpha +1,\beta +1)}}} = \sqrt{\frac{\kappa ^{(\alpha ,\beta )}_{j}}{\lambda ^{(\alpha ,\beta )}_j \kappa ^{(\alpha +1,\beta +1)}_{j-1}} } y_{j-1}. \end{aligned}$$

Using (A.5) and (A.6), we now get that

$$\begin{aligned} \Vert p \Vert ^2_{L^2_{\nu ^{(\alpha ,\beta )}}} \ge \sum ^{M}_{j=1} \frac{|y_{j-1}|^2}{ \lambda ^{(\alpha ,\beta )}_j \kappa ^{(\alpha +1,\beta +1)}_{j-1}} \ge \frac{1}{\lambda ^{(\alpha ,\beta )}_{M} } \Vert p' \Vert ^2_{L^2_{\nu ^{(\alpha +1,\beta +1)}}}, \end{aligned}$$

as required. $\square$

We also require several results concerning the asymptotic behavior of Jacobi polynomials. The first is as follows (see [31, Thm. 7.32.1]):

$$\begin{aligned} \Vert P^{(\alpha ,\beta )}_j \Vert _{L^\infty } = \mathcal {O}\left( j^{q} \right) ,\quad n \rightarrow \infty ,\qquad q = \max \{ \alpha ,\beta , -1/2 \} . \end{aligned}$$

Hence, using (A.2), we find that the normalized functions $\phi _j$ defined by (2.6) satisfy

$$\begin{aligned} \Vert \phi _j \Vert _{L^\infty } = \mathcal {O}\left( j^{q+1/2} \right) ,\quad j \rightarrow \infty , \end{aligned}$$

(A.7)

which gives (2.7). We also note the following local estimates for Jacobi polynomials. If $k=0,1,2,\ldots$ and $c >0$ is a fixed constant, then

$$\begin{aligned} \left| \frac{\,\mathrm {d}^kP^{(\alpha ,\beta )}_j(t)}{\,\mathrm {d}t^k} \Bigg |_{t = \cos \theta } \right| = \left\{ \begin{array}{ll} \theta ^{-\alpha -k-1/2} \mathcal {O}\left( j^{k-1/2} \right) , &{} c j^{-1} \le \theta \le \pi /2, \\ \mathcal {O}\left( j^{2k+\alpha } \right) , &{} 0 \le \theta \le c j^{-1}, \end{array} \right. \end{aligned}$$

(A.8)

as $j \rightarrow \infty$. See [31, Thm. 7.32.4]. This estimate bounds the Jacobi polynomial and its derivatives for $0 \le t \le 1$. For negative t, we may use the relation

$$\begin{aligned} P^{(\alpha ,\beta )}_j(-t) = (-1)^j P^{(\beta ,\alpha )}_j(t). \end{aligned}$$

(A.9)

Hence behavior of $P^{(\alpha ,\beta )}_j(t)$ and its derivatives for $t<0$ is given by (A.8) with $\alpha$ replaced by $\beta$.