Geometrically designed, variable knot regression splines

Abstract

A new method of Geometrically Designed least squares (LS) splines with variable knots, named GeDS, is proposed. It is based on the property that the spline regression function, viewed as a parametric curve, has a control polygon and, due to the shape preserving and convex hull properties, it closely follows the shape of this control polygon. The latter has vertices whose x-coordinates are certain knot averages and whose y-coordinates are the regression coefficients. Thus, manipulation of the position of the control polygon may be interpreted as estimation of the spline curve knots and coefficients. These geometric ideas are implemented in the two stages of the GeDS estimation method. In stage A, a linear LS spline fit to the data is constructed, and viewed as the initial position of the control polygon of a higher order ($n>2$) smooth spline curve. In stage B, the optimal set of knots of this higher order spline curve is found, so that its control polygon is as close to the initial polygon of stage A as possible and finally, the LS estimates of the regression coefficients of this curve are found. The GeDS method produces simultaneously linear, quadratic, cubic (and possibly higher order) spline fits with one and the same number of B-spline coefficients. Numerical examples are provided and further supplemental materials are available online.

Appendices

Appendices

Proofs of the results of section 3.1

Proof of Theorem 3.4

Note that, for $n=2$, $\xi _i\equiv \xi _i^*$, $i=1,\ldots ,p$, hence $V^a[g]\equiv V[g]$ and the bound in (23), which is zero, is sharp. For $n>2$, from (4) it follows that $\xi _1^*\equiv a\equiv \xi _1$ and $\xi _p^*\equiv b\equiv \xi _p$, and from the definitions of V[g] and $V^a[g]$, (9) and (22) respectively, we have

$$\begin{aligned} \Vert V[g]-V^a[g]\Vert _{\infty }= & {} \text{ max }_{t\in [a,b]}\left| \sum _{i=1}^p\left( g\left( \xi _i^*\right) -g\left( \xi _i\right) \right) N_{i,n}(t)\right| \nonumber \\\le & {} \text{ max }_{t\in [a,b]}\sum _{i=1}^p\left| \left( g\left( \xi _i^*\right) -g\left( \xi _i\right) \right) \right| N_{i,n}(t)\nonumber \\\le & {} \text{ max }_{t\in [a,b]}\sum _{i=1}^p\left\{ \text{ max }_{j\in \{2,\ldots ,p-1\}} \left| \left( g\left( \xi _j^*\right) -g\left( \xi _j\right) \right) \right| \right\} N_{i,n}(t)\nonumber \\\le & {} \text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \left( g\left( \xi _j^*\right) -g(\xi _j)\right) \right| \text{ max }_{t\in [a,b]}\sum _{i=1}^pN_{i,n}(t)\nonumber \\= & {} \text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \left( g\left( \xi _j^*\right) -g(\xi _j)\right) \right| , \end{aligned}$$

(27)

where the last equality follows from the partition of unity property of B-splines (see Sect. 2). Applying the definition of the modulus of continuity to (27) we have

$$\begin{aligned} \Vert V[g]-V^a[g]\Vert _{\infty }\le & {} \text{ max }_{j\in \{2,\ldots ,p-1\}} \left| \left( g\left( \xi _j^*\right) -g(\xi _j)\right) \right| \nonumber \\\le & {} \omega \left( g;\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \xi _j^*-\xi _j\right| \right) . \end{aligned}$$

(28)

From the definition (4) of the Greville sites $\xi _i^*$ we have $\xi _j^*=(t_{j+1}+\ldots + t_{j+n-1})/(n-1)$, $j=2,\ldots ,p-1$. From (21), it follows that $t_{j+1}=\left( \xi _{j-(n-2)}+\ldots +\xi _j \right) /(n-1),\ldots , t_{j+n-1}=\left( \xi _{j}+\ldots +\xi _{j+(n-2)} \right) /(n-1)$, where we have defined $\xi _{1-l}:=a$ and $\xi _{p+l}:=b$, $l=1,2,\ldots$. Consider the $\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \xi _j-\xi _j^*\right|$ and assume it is achieved for some $j^m$, $2\le j^m<p-1$. Expressing $\xi _{j^m}^*$ in terms of $\xi _{j^m}$, using the above equalities, after some algebra, it is not difficult to see that

$$\begin{aligned} \left| \xi _{j^m}-\xi _{j^m}^*\right| =\frac{1}{(n-1)^2} \left| \sum _{i=1}^{n-2}i\left( \xi _{j^m+(n-1-i)}+\xi _{j^m-(n-1-i)}\right) -(n-1)(n-2) \xi _{j^m}\right| \end{aligned}$$

(29)

and if we now rearrange the terms in the sum in (29), we obtain

$$\begin{aligned} \left| \xi _{j^m}-\xi _{j^m}^*\right| =\frac{1}{(n-1)^2}\left| \sum _{i=1}^{n-2}i\left( \left( \xi _{j^m+(n-1-i)}-\xi _{j^m}\right) -\left( \xi _{j^m}-\xi _{j^m-(n-1-i)}\right) \right) \right| . \end{aligned}$$

(30)

Assume that $\sum _{i=1}^{n-2}i\left( \xi _{j^m+(n-1-i)}-\xi _{j^m}\right) >\sum _{i=1}^{n-2}i\left( \xi _{j^m}-\xi _{j^m-(n-1-i)}\right)$. In this case, it is not difficult to see that (30) is bounded by

$$\begin{aligned} \left| \xi _{j^m}-\xi _{j^m}^*\right|\le & {} \frac{1}{(n-1)^2}\sum _{i=1}^{n-2}i \left( \xi _{j^m+(n-1-i)}-\xi _{j^m}\right) \nonumber \\\le & {} \,\frac{1}{(n-1)^2}\frac{(n-2)(n-1)}{2}\left( \xi _{j^m+(n-2)}-\xi _{j^m} \right) \nonumber \\\le & {} \,\frac{(n-2)}{2(n-1)}\left( \xi _{j^m+(n-2)}-\xi _{j^m}\right) \nonumber \\\le & {} \,\frac{(n-2)^2}{2(n-1)}\text{ max }_{j\in \{1,\ldots ,p-1\}}(\xi _{j+1}-\xi _j). \end{aligned}$$

(31)

Similarly, it can be shown that if $\sum _{i=1}^{n-2}i\left( \xi _{j^m+(n-1-i)}-\xi _{j^m}\right) \le \sum _{i=1}^{n-2}i\Big (\xi _{j^m}- \xi _{j^m-(n-1-i)}\Big )$ the bound in (31) also holds. Thus, from (31) and (28) we have

$$\begin{aligned} \Vert V[g]-V^a[g]\Vert _{\infty }\le \omega \left( g;\frac{(n-2)^2}{2(n-1)} \text{ max }_{j\in \{1,\ldots ,p-1\}}(\xi _{j+1}-\xi _j)\right) . \end{aligned}$$

(32)

Using the monotonicity and subadditivity of $\omega (g;h)$ in h, from (32) we finally obtain

$$\begin{aligned} \Vert V[g]-V^a[g]\Vert _{\infty }\le \left\lceil \frac{(n-2)^2}{2(n-1)}\right\rceil \omega \left( g;\text{ max }_{j\in \{1, \ldots ,p-1\}}(\xi _{j+1}-\xi _j)\right) \end{aligned}$$

where $\lceil \nu \rceil :=\hbox {min}\{z\in \mathbb {Z}:\nu \le z\}$. This completes the proof of Theorem 3.4. $\square$

Proof of Corollary 3.5

This follows directly from (32) and from the definition, (24) of $\omega (g;h)$, i.e.

$$\begin{aligned} \Vert V[t]-V^a[t]\Vert _{\infty }=\left\| t-\sum _{i=1}^p\delta _{i+1} N_{i,n}(t)\right\| _{\infty }\le \frac{(n-2)^2}{2(n-1)} \text{ max }_{j\in \{1,\ldots ,p-1\}}(\delta _{j+2}-\delta _{j+1}). \end{aligned}$$

$\square$

Proof of Corollary 3.6

From (27), for $n=3$ and $g=\hat{f}$, we have

$$\begin{aligned} \Vert V[\hat{f}]-V^a[\hat{f}]\Vert _{\infty }\le & {} \text{ max }_{j\in \{2,\ldots ,p-1\}} \left| \hat{f}\left( \varvec{\delta }_{l,2},\varvec{\hat{\alpha }};\xi _j^*\right) -\hat{f}\left( \varvec{\delta }_{l,2},\varvec{\hat{\alpha }};\delta _{j+1}\right) \right| \end{aligned}$$

(33)

$$\begin{aligned}= & {} \text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \sum _{i=1}^p\hat{\alpha }_i N_{i,2}\left( \xi _j^*\right) -\sum _{i=1}^p\hat{\alpha }_i N_{i,2} (\delta _{j+1})\right| \nonumber \\= & {} \text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \sum _{i=1}^p\hat{\alpha }_i N_{i,2}\left( \xi _j^*\right) -\hat{\alpha }_j\right| \nonumber \\= & {} \text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \sum _{i=j-1}^{j+1}\hat{\alpha }_i N_{i,2}\left( \xi _j^*\right) -\hat{\alpha }_j\right| \end{aligned}$$

(34)

Recall that $n=3$ and hence, $\xi _j^*=(t_{j+1}+t_{j+2})/2$, and $t_{j+1}=(\delta _j+\delta _{j+1})/2$, $t_{j+2}=(\delta _{j+1}+\delta _{j+2})/2$. Therefore, we need to consider the cases when $\delta _{j}<\xi _j^*\le \delta _{j+1}$, or $\delta _{j+1}\le \xi _j^*<\delta _{j+2}$, $2\le j\le p-1$. In the first case, applying the Mansfield-De Boor-Cox recurrence formula we know that if $\delta _j<\xi _j^*<\delta _{j+1}$, then $\sum _{i=j-1}^{j+1}\hat{\alpha }_i N_{i,2}\left( \xi _j^*\right) =\hat{\alpha }_{j-1} N_{j-1,2}\left( \xi _j^*\right) +\hat{\alpha }_{j}N_{j,2}\left( \xi _j^*\right)$, which is a convex combination of only two B-spline coefficients. Thus, (34) becomes

$$\begin{aligned}&\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \hat{\alpha }_{j-1}N_{j-1,2} \left( \xi _j^*\right) +\hat{\alpha }_j N_{j,2}\left( \xi _j^*\right) -\hat{\alpha }_j\right| \nonumber \\&\quad =\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| \hat{\alpha }_{j-1} \frac{\delta _{j+1}-\xi _j^*}{\delta _{j+1}-\delta _{j}} +\hat{\alpha }_j\frac{\xi _j^*-\delta _{j}}{\delta _{j+1}-\delta _{j}} -\hat{\alpha }_j\frac{\delta _{j+1}-\delta _{j}}{\delta _{j+1} -\delta _{j}}\right| \nonumber \\&\quad =\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| (\hat{\alpha }_{j-1} -\hat{\alpha }_j)\right| \left( \frac{\delta _{j+1}-\xi _j^*}{\delta _{j+1} -\delta _{j}}\right) \nonumber \\&\quad <\text{ max }_{j\in \{2,\ldots ,p-1\}}\left| (\hat{\alpha }_{j-1} -\hat{\alpha }_j)\right| \left( \frac{\frac{1}{4}\left( \delta _{j+1} -\delta _{j}\right) }{\delta _{j+1}-\delta _{j}}\right) \nonumber \\&\quad =\frac{1}{4}\text{ max }_{j\in \{2,\ldots ,p-1\}} \left| (\hat{\alpha }_{j-1}-\hat{\alpha }_j)\right| , \end{aligned}$$

(35)

where we have used the fact that $\delta _{j+2}-\delta _{j+1}>0$ to arrive at the last inequality. Similarly, it is not difficult to see that the same bound as in (35) holds in the case when $\delta _{j+1}\le \xi _j^*\le \delta _{j+2}$. This completes the proof of Corollary 3.6. $\square$