Variable Selection in Multivariate Functional Linear Regression

Abstract

Multivariate functional linear regression is commonly adopted to model the effects of several function-valued covariates on a scalar response. To select functional covariates with a time-varying effect, we develop a framework based on the reproducing kernel Hilbert space (RKHS). In particular, each coefficient function is assumed to reside in this RKHS and an RKHS norm is chosen as the penalty function in the regularized empirical risk function. This special penalty term enables us to achieve sparsity and smoothness when fitting multivariate functional linear models. Moreover, simulation studies demonstrate that the proposed estimator compares favorably with some traditional methods in variable selection, function estimation and prediction in finite samples. Finally, we apply the proposed framework to two real examples.

Appendix

1.1 Appendix A.1. Proof of Proposition 1

Proof

Let $\xi _{ij}(t) = (K_1X_{ij})(t)$ for $t \in [0, 1]$ and $\mathop {}\!\mathcal {L}(\varvec{\xi }_j)$ denote the linear space expanded by $\xi _{ij}, i = 1, \ldots , n$. Based on [26], we know that $\xi _{ij} \in \mathop {}\!\mathcal {H}_1$, and hence $\mathop {}\!\mathcal {L}(\varvec{\xi }_j)$ is a subspace of $\mathop {}\!\mathcal {H}_1$. Each $\beta _j$ can be written as

$$\begin{aligned} \beta _j = d_j + \sum _{i = 1}^n c_{ij}\xi _{ij} + \rho _j \end{aligned}$$

for some $\rho _j \in \mathop {}\!\mathcal {H}_1 \ominus \mathop {}\!\mathcal {L}(\varvec{\xi }_j)$, where $\mathop {}\!\mathcal {H}_1 \ominus \mathop {}\!\mathcal {L}(\varvec{\xi }_j)$ denotes the orthogonal complement of $\mathop {}\!\mathcal {L}(\varvec{\xi }_j)$ in $\mathop {}\!\mathcal {H}_1$.

Next, we plug the expression of $\beta _j$ into (4). Let $u_i = Y_i - \sum _{j = 1}^p \int _{\mathop {}\!\mathcal {I}_j} d_jX_{ij}(t) \textrm{d}t$ for $i = 1, \ldots , n$. Then $\hat{\bar{\beta }}_j$’s minimize

$$\begin{aligned} Q(\bar{\beta }_1,\ldots , \bar{\beta }_j)&= \frac{1}{n}\sum _{i = 1}^n \left\{ u_i - \sum _{j = 1}^p \int _{\mathop {}\!\mathcal {I}_j} X_{ij}(t)\bar{\beta }_j(t) \textrm{d}t \right\} ^2 + \tau \sum _{j = 1}^p\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}} \\&= \frac{1}{n}\sum _{i = 1}^n \left\{ u_i - \sum _{j = 1}^p \int _{\mathop {}\!\mathcal {I}_j} X_{ij}(t) \langle K_1(\cdot , t), \bar{\beta }_j \rangle _{\mathop {}\!\mathcal {H}_1} \textrm{d}t \right\} ^2 + \tau \sum _{j = 1}^p\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}} \\&= \frac{1}{n}\sum _{i = 1}^n \left\{ u_i - \sum _{j = 1}^p \int _{\mathop {}\!\mathcal {I}_j} X_{ij}(t) \left\langle K_1(\cdot , t), \sum _{i = 1}^n c_{ij}\xi _{ij} + \rho _j \right\rangle _{\mathop {}\!\mathcal {H}_1} \textrm{d}t \right\} ^2 \\&\quad + \tau \sum _{j = 1}^p\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}} \\&= \frac{1}{n}\sum _{i = 1}^n \left( u_i - \sum _{j = 1}^p \sum _{l = 1}^n c_{lj} \langle \xi _{ij}, \xi _{lj} \rangle _{\mathop {}\!\mathcal {H}_1} - \sum _{j = 1}^p \langle \xi _{ij}, \rho _j \rangle _{\mathop {}\!\mathcal {H}_1}\right) ^2 \\&\quad + \tau \sum _{j = 1}^p\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}} \\&= \frac{1}{n}\sum _{i = 1}^n \left( u_i - \sum _{j = 1}^p \sum _{l = 1}^n c_{lj} \langle \xi _{ij}, \xi _{lj} \rangle _{\mathop {}\!\mathcal {H}_1}\right) ^2 + \tau \sum _{j = 1}^p\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}} \end{aligned}$$

As we know that $\Vert \bar{\beta }_j\Vert _{\mathop {}\!\mathcal {H}_1}^2 = \Vert \sum _{i = 1}^n c_{ij}\xi _{ij}\Vert _{\mathop {}\!\mathcal {H}_1}^2 + \Vert \rho _j\Vert _{\mathop {}\!\mathcal {H}_1}^2$, the penalty term is minimized when $\rho _j = 0$. The proof is completed. □