Varying coefficient partially functional linear regression models

Abstract

By relaxing the linearity assumption in partial functional linear regression models, we propose a varying coefficient partially functional linear regression model (VCPFLM), which includes varying coefficient regression models and functional linear regression models as its special cases. We study the problem of functional parameter estimation in a VCPFLM. The functional parameter is approximated by a polynomial spline, and the spline coefficients are estimated by the ordinary least squares method. Under some regular conditions, we obtain asymptotic properties of functional parameter estimators, including the global convergence rates and uniform convergence rates. Simulation studies are conducted to investigate the performance of the proposed methodologies.

Appendix: Proofs of Theorems

Denote $B_{js}=K_{jn}^{1/2}\phi _{js}$, where $\phi _{js}$’s are the normalized B-splines in the space $S_{k_j,N_{jn}}$ for $s=1,\ldots , K_{jn}$ and $j=0,1,\ldots , p$. It follows from Theorem 4.2 of Chapter 5 in DeVore and Lorentz (1993) that for any spline function $\sum _{s=1}^{K_{jn}} b_{js} B_{js}$, there are positive constants $M_1$ and $M_2$ such that

$$\begin{aligned} M_1|b_j|_2^2\le \int \left\{ \sum _{s=1}^{K_{jn}} b_{js} B_{js}\right\} ^2\le M_2|b_j|_2^2, \end{aligned}$$

(8)

where $|\cdot |_2$ is Euclidean norm.

Define $\mathbf {B}=(\mathbf {X},\mathbf {Z})$, where

$$\begin{aligned} \mathbf {X}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c} <X_1,B_{01}> &{}<X_1,B_{02}>&{}\dots &{}<X_1,B_{0K_{0n}}> \\ <X_2,B_{01}> &{}<X_2,B_{02}>&{}\dots &{}<X_2,B_{0K_{0n}}> \\ \vdots &{} \vdots &{}\ddots &{}\vdots \\ <X_n,B_{01}> &{}<X_n,B_{02}>&{}\dots &{}<X_n,B_{0K_{0n}}> \end{array}\right) , \end{aligned}$$

$$\begin{aligned} \mathbf {Z}=\left( \begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} B_{11}(U_1)Z_{11}&{}\dots &{} B_{1K_{1n}}(U_1)Z_{11}&{}\dots &{} B_{p1}(U_1)Z_{1p}&{}\dots &{}B_{pK_{pn}}(U_1)Z_{1p}\\ B_{11}(U_2)Z_{21}&{}\dots &{} B_{1K_{1n}}(U_2)Z_{21}&{}\dots &{} B_{p1}(U_2)Z_{2p}&{}\dots &{}B_{pK_{pn}}(U_2)Z_{2p}\\ \vdots &{} \ddots &{}\vdots &{}\ddots &{}\vdots &{}\ddots &{}\vdots \\ B_{11}(U_n)Z_{n1}&{}\dots &{} B_{1K_{1n}}(U_n)Z_{n1}&{}\dots &{} B_{p1}(U_n)Z_{np}&{}\dots &{}B_{pK_{pn}}(U_n)Z_{np} \end{array}\right) . \end{aligned}$$

To prove Theorems 1 and 2, we require the following Lemmas.

Lemma 1

If Assumptions (A1)–(A6) hold, we have

$$\begin{aligned} \sup _{\begin{array}{c} a_j\in S_{k_j,N_{jn}}\\ j=0,1,\ldots ,p \end{array}}\Big | \frac{\frac{1}{n}\sum _{i=1}^n \left\{ <X_i,a_0>+\sum _{j=1}^p a_j(U_i)Z_{ij}\right\} ^2}{E(<X_1,a_0>+\sum _{j=1}^p a_j(U_1)Z_{1j})^2}-1\Big |=o_p(1). \end{aligned}$$

Proof

For an i.i.d. random variable sequence $\xi _1,\ldots ,\xi _n$, let $E_n(\xi _i)=\frac{1}{n}\sum _{i=1}^n\xi _i$. By Assumptions (A2)–(A5), we have

$$\begin{aligned} E\left( <X_1,a_0>\!+\!\sum \limits _{j=1}^p a_j(U_1)Z_{1j}\!\right) ^{\!2}\!\asymp \! E\left( \!<X_1,a_0>^2+\sum \limits _{j=1}^p a_j^2(U_1)\!\right) \!\asymp \! \sum _{j=0}^p||a_j||_2^2. \end{aligned}$$

Consequently, we only need to prove that for arbitrary given $\eta >0$, as $n\rightarrow \infty$, we have

$$\begin{aligned} \mathbb {I}\!=\!P\left\{ \sup _{\begin{array}{c} a_j\in S_{k_j,N_{jn}}\\ j=0,1,\ldots ,p \end{array}} \frac{\left| \left( E_n-E\right) \left[ <X_i,a_0>+\sum _{j=1}^p a_j(U_i)Z_{ij}\right] ^2\right| }{\sum _{j=0}^p||a_j||_2^2}\!>\!(p+1)\eta \right\} \!\rightarrow \! 0. \end{aligned}$$

If $|(E_n\!-\!E)<X_i,a_0>^2|\!\le \! \eta ||a_0||_2^2$, $|(E_n-E)\left\{ <X_i,a_0\!>\!a_j(U_i)Z_{ij}\right\} |\!\le \! \eta ||a_0||_2||a_j||_2$ for $j\!=\!1,\ldots ,p$, and $|(E_n-E)\{a_j(U_i)a_{j'}(U_i)Z_{ij}Z_{ij'}\}|\!\le \! \eta ||a_j||_2||a_{j'}||_2$ for $j',j=1,\ldots ,p$, we obtain

$$\begin{aligned} |\left( E_n\!-\!E\right) \left\{ <X_i,a_0\!>\!+\sum \limits _{j=1}^p a_j(U_i)Z_{ij}\!\right\} ^2 \!|\le \eta \left( \sum \limits _{j=0}^p ||a_j||_2)^2\!\le \! (p\!+\!1\!\right) \eta \sum \limits _{j=0}^p ||a_j||_2^2. \end{aligned}$$

Thus, we have

$$\begin{aligned} \mathbb {I}&\le P\left\{ \sup _{a_0\in S_{k_0,N_{jn}}} \frac{|(E_n-E)<X_i,a_0>^2|}{||a_0||_2^2} >\eta \right\} \nonumber \\&\quad \ +\, 2\sum _{j=1}^p P\left\{ \sup _{{\begin{array}{c} a_j\in S_{k_j,N_{jn}}\\ a_{0}\in S_{k_{0},N_{0n}} \end{array}}} \frac{|(E_n-E)[<X_i,a_0>a_j(U_i)Z_{ij}]|}{||a_0||_2||a_j||_2}>\eta \right\} \nonumber \\&\quad \ +\, \sum _{j=1}^p\sum _{j'=1}^p P\left\{ \sup _{\begin{array}{c} a_j\in S_{k_j,N_{jn}}\\ a_{j'}\in S_{k_{j'},N_{j'n}} \end{array}} \frac{|(E_n-E)[a_j(U_i)a_{j'}(U_i)Z_{ij}Z_{ij'}]|}{||a_j||_2||a_{j'}||_2}>\eta \right\} \nonumber \\&\mathop {=}\limits ^{\Delta } I_1+2I_2+I_3. \end{aligned}$$

For $I_1$, it follows from Lemma 5.2 of Cardot et al. (1999) that $I_1\rightarrow 0$ as $n\rightarrow \infty$. Following the similar argument of Lemma 1 in Huang and Shen (2004), it is easily shown that $I_3\rightarrow 0$ as $n\rightarrow \infty$. Consequently, we only need to prove that for $j=1,\cdots ,p$,

$$\begin{aligned} \mathbb {I}_j=P\left\{ \sup _{\begin{array}{c} a_j\in S_{k_j,N_{jn}}\\ a_{0}\in S_{k_0,N_{0n}} \end{array}} \frac{|(E_n-E)[<X_i,a_0>a_j(U_i)Z_{ij}]|}{||a_0||_2||a_j||_2}>\eta \right\} \rightarrow 0~~\mathrm{as}~~ n\rightarrow \infty . \end{aligned}$$

Note that $<X_i,a_0>a_j(U_i)Z_{ij}=\sum _{s_0=1}^{K_{0n}}\sum _{s_j=1}^{K_{jn}} b_{0s_0}b_{js_j}<X_i,B_{0s_0}>B_{js_j}(U_i)Z_{ij}$ for $j=1,\ldots ,p$. Hence, if $|(E_n-E)\{<X_i,B_{0s_0}>B_{js_j}(U_i)Z_{ij}\}|\le \eta$ for $s_0=1,\ldots ,K_{0n}$ and $s_j=1,\ldots ,K_{jn}$, it follows from the Cauchy-Schwarz inequality and Eq. (8) that

$$\begin{aligned} |(E_n-E)[<X_i,a_0>a_j(U_i)Z_{ij}]|\le & {} \eta \sum _{s_0=1}^{K_{0n}}\sum _{s_j=1}^{K_{jn}}|b_{0s_0}||b_{js_j}|\nonumber \\\le & {} K_{0n}^{1/2}K_{jn}^{1/2}(\sum _{s_0=1}^{K_{0n}} b_{0s_0}^2)^{1/2}(\sum _{s_j=1}^{K_{jn}}b_{js_j}^2)^{1/2}\nonumber \\\le & {} C\eta \mathcal {K}_n||a_0||_2||a_j||_2. \end{aligned}$$

Thus, we have

$$\begin{aligned} \mathbb {I}_j\le \sum \limits _{s_0=1}^{K_{0n}}\sum \limits _{s_j=1}^{K_{jn}}P\left\{ |(E_n-E)\left( <X_i,B_{0s_0}>B_{js_j}(U_i)Z_{ij}\right) |>\eta /C\mathcal {K}_n\right\} . \end{aligned}$$

(9)

Denote $\tilde{Z}_{ij}=Z_{ij}I(|Z_{ij}|\le n^{\delta })$ for $j=1,\ldots ,p$, and we assume $m_0>\delta ^{-1}$ with $\delta >0$. It follows from condition (A6) that as $n\rightarrow \infty$, we have

$$\begin{aligned}P\{\exists ~i=1,\ldots ,n~\text {such~that}~Z_{ij}\ne \tilde{Z}_{ij}\}\le \sum _{i=1}^n P\{|Z_{ij}|>n^{\delta }\}\nonumber \le \frac{E|Z_{1j}|^{m_0}}{n^{m_0\delta -1}}\rightarrow 0. \end{aligned}$$

Combining condition (A1) and Eq. (9) yields

$$\begin{aligned} \mathbb {I}_j\lesssim n^{2r}\max _{\begin{array}{c} s_0=1,\ldots ,K_{0n},\\ s_j=1,\ldots ,K_{jn} \end{array}}P\left\{ |(E_n-E)\left( <X_i,B_{0s_0}>B_{js_j}(U_i)Z_{ij}\right) |>\eta /C\mathcal {K}_n\right\} . \end{aligned}$$

From Lemma A.8 of Ferraty and Vieu (2006), we have

$$\begin{aligned}\mathbb {I}_j\lesssim n^{2r} \exp (-C\eta ^2 n^{1-(2\delta +3r)}).\end{aligned}$$

Since $\delta ^{-1}<m_0$ and $0<r<1/3$, we can always find $\delta >0$ and $r>0$ such that $2\delta +3r<1$. Hence, as $n\rightarrow \infty$, we have $\mathbb {I}_j\rightarrow 0$ for $j=1,\ldots ,p$. Combining the above equations leads to Lemma 1. $\square$

Lemma 2

If Assumptions (A1)–(A6) hold, there is an interval $[M_3, M_4]$ with $0<M_3<M_4$ such that as $n\rightarrow \infty$, we have

$$\begin{aligned} P\Big \{\text {all the eigenvalues of}~\frac{\mathbf {B}^T\mathbf {B}}{n}~\text {fall in}~ [M_3,M_4]\Big \}\rightarrow 1. \end{aligned}$$

Proof

The proof of Lemma 2 is similar to that given in Lemma 2 of Huang and Shen (2004). Hence, we here omit it. $\square$

Lemma 2 shows that the convergence rate of estimator $\widehat{b}$ does not depend on the eigenvalues of the covariance operator $\Gamma$ of $X$. Thus, it follows from Cardot et al. (2003) that the convergence rate of our proposed estimator can attain the nonparametric convergence rate.

Proof of Theorem 1

Denote $\tilde{Y}_i=<X_i,a_0>+\sum _{j=1}^p a_j(U_i)Z_{ij}$ and $\tilde{Y}=(\tilde{Y}_1,\cdots ,\tilde{Y}_n)^T$. Let $\tilde{b}=(\mathbf {B}^T\mathbf {B})^{-1}\mathbf {B}^T\tilde{Y}$, where $\tilde{b}=(\tilde{b}_0^T,\tilde{b}_1^T,\ldots ,\tilde{b}_p^T)^T$ with $\tilde{b}_j=(\tilde{b}_{j1},\ldots ,\tilde{b}_{jK_{jn}})^T$ for $j=0,1,\ldots ,p$. Denote $\tilde{a}_j=\sum _{s=1}^{K_{jn}} \tilde{b}_{js}B_{js}$ and $\varepsilon =(\varepsilon _1,\cdots ,\varepsilon _n)^T$. Under the above notation, it follows from Lemma 2 that $E|\widehat{b}-\tilde{b}|^2=E(\varepsilon ^T\mathbf {B}(\mathbf {B}^T\mathbf {B})^{-1}(\mathbf {B}^T\mathbf {B})^{-1}\mathbf {B}^T\varepsilon ) =\frac{\sigma ^2}{n}E(\mathrm{tr}(\frac{1}{n}\mathbf {B}^T\mathbf {B})^{-1})\lesssim \mathcal {K}_n/n$. Hence, it follows from Eq. (8) that

$$\begin{aligned} \sum _{j=0}^p ||\widehat{a}_j-\tilde{a}_j||_2^2\asymp |\widehat{b}-\tilde{b}|^2=O_p(\frac{\mathcal {K}_n}{n}). \end{aligned}$$

(10)

Again, it follows from condition (A1) and Theorem XII.1 of de Boor (2001) that for $j=0,1,\ldots ,p$, there exist spline function $a_j^*\in S_{k_j,N_{jn}}$ and constant $C_j>0$ such that

$$\begin{aligned} ||a_j^*-a_j||_{\infty }\le C_jh_j^q\lesssim \mathcal {K}_n^{-q}. \end{aligned}$$

(11)

Let $b^*=(b_0^{*T},b_1^{*T},\ldots ,b_p^{*T})^T$ with $b_j^*=(b_{j1}^*,\ldots ,b_{jK_{jn}}^*)^T$, and $a_j^*=\sum _{s=1}^{K_{jn}}b_{js}^*B_{js}$ for $j=0,1,\ldots ,p$. It follows from Equation (8) and Lemma 2 that $\sum _{j=0}^p || a_j^*-\tilde{a}_j||_2^2\asymp |b^*-\tilde{b}|^2\asymp \frac{1}{n}(\tilde{b}-b^*)^T\mathbf {B}^T\mathbf {B}(\tilde{b}-b^*)$ a.s.. Since $\mathbf {B}(\mathbf {B}^T\mathbf {B})^{-1}\mathbf {B}^T$ is an orthogonal projection matrix, we have

$$\begin{aligned} \frac{1}{n}(\tilde{b}-b^*)^T\mathbf {B}^T\mathbf {B}(\tilde{b}-b^*)\le & {} \frac{1}{n}|\tilde{Y}-\mathbf {B}b^*|^2\\\le & {} \frac{1}{n}\sum _{i=1}^n\{<X_i,a_0-a_0^*>+\sum _{j=1}^p[a_j(U_i)-a_j^*(U_i)]Z_{ij}\}^2. \end{aligned}$$

By Assumptions (A2)–(A4) and Eq. (11), we obtain

$$\begin{aligned} E\{<X_1,a_0-a_0^*>+\sum _{j=1}^p(a_j(U_1)\!-\!a_j^*(U_1))Z_{1j}\}^2\asymp \sum _{j=0}^p||a_j\!-\!a_j^*||_2^2=O_p(\mathcal {K}_n^{-2q}).\nonumber \\ \end{aligned}$$

(12)

For $j=0,1,\ldots ,p$, we can obtain

$$\begin{aligned} ||\widehat{a}_j-a_j||_2^2\le 3(||\widehat{a}_j-\tilde{a}_j||_2^2+||\tilde{a}_j-a_j^*||_2^2+||a_j^*-a_j||_2^2). \end{aligned}$$

(13)

Combining Eqs. (10)–(13) yields Theorem 1. $\square$

Proof of Theorem 2

For $j=0,1,\ldots ,p$, we have

$$\begin{aligned} ||\widehat{a}_j-a_j||_{\infty }\le ||\widehat{a}_j-\tilde{a}_j||_{\infty }+||\tilde{a}_j-a_j^*||_{\infty }+||a_j^*-a_j||_{\infty }, \end{aligned}$$

(14)

where $\tilde{a}_j$ and $a_j^*$ are defined in the proof of Theorem 1. Also, it follows from Huang et al. (2004) that there is a constant $M>0$ such that

$$\begin{aligned} ||g_j||_{\infty }\le M\sqrt{K_{jn}}||g_j||_2 \end{aligned}$$

(15)

for $g_j\in S_{k_j,N_{jn}}$ ($j=0,1,\ldots ,p)$. Hence, by condition (A1), (10), (13) and (15), we obtain

$$\begin{aligned} ||\widehat{a}_j-\tilde{a}_j||_{\infty }\le M\sqrt{K_{jn}}||\widehat{a}_j-\tilde{a}_j||_2=O_p(\mathcal {K}_n n^{-1/2}), \end{aligned}$$

$$\begin{aligned} ||\tilde{a}_j-a_j^*||_{\infty }\le M\sqrt{K_{jn}}||\tilde{a}_j-a_j^*||_2=O_p(\mathcal {K}_n^{1/2-q}). \end{aligned}$$

Again, it follows from Eq. (11) that $||a_j^*-a_j||_{\infty }= O(\mathcal {K}_n^{-q})=o(\mathcal {K}_n^{1/2-q})$. Therefor, combining the above equations leads to Theorem 2. $\square$