Penalised spline estimation for generalised partially linear single-index models

Abstract

Generalised linear models are frequently used in modeling the relationship of the response variable from the general exponential family with a set of predictor variables, where a linear combination of predictors is linked to the mean of the response variable. We propose a penalised spline (P-spline) estimation for generalised partially linear single-index models, which extend the generalised linear models to include nonlinear effect for some predictors. The proposed models can allow flexible dependence on some predictors while overcome the “curse of dimensionality”. We investigate the P-spline profile likelihood estimation using the readily available R package mgcv, leading to straightforward computation. Simulation studies are considered under various link functions. In addition, we examine different choices of smoothing parameters. Simulation results and real data applications show effectiveness of the proposed approach. Finally, some large sample properties are established.

Appendix

Identifiability and reparametrisation

For model identifiability, the single-index parameter is constrained such that $||{\varvec{\theta }}|| =1$ and the first element $\theta _1>0$. Maximising (3) is actually a constrained (penalised) maximum likelihood problem. We reparameterise ${\varvec{\theta }}$ to handle this constraint. Let ${\varvec{\zeta }}=(\zeta _1,\ldots ,\zeta _{s-1})^{\mathsf{T}}$ and define ${\varvec{\theta }}=(1,{\varvec{\zeta }}^{\mathsf{T}})^{\mathsf{T}}/{\sqrt{1+||{\varvec{\zeta }}||^2}}$. Now the reparametrised parameter ${\varvec{\zeta }}$ is unconstrained. The Jacobian matrix of $\partial {\varvec{\theta }}/ \partial {\varvec{\zeta }}$ is

$$\begin{aligned} \frac{\partial {\varvec{\theta }}}{\partial {{\varvec{\zeta }}}}= & {} -\left( 1+\Vert {\varvec{\zeta }}\Vert ^2\right) ^{-\frac{3}{2}}\nonumber \\&\times \left[ \begin{array}{cccc} \zeta _1 &{}\quad \zeta _2 &{}\quad \cdots &{}\quad \zeta _{s-1} \\ -(1+\Vert {\varvec{\zeta }}\Vert ^2)+\zeta ^2_1 &{}\quad \zeta _2 \zeta _1 &{}\quad \cdots &{}\quad \zeta _{s-1} \zeta _1 \\ \zeta _1 \zeta _2 &{}\quad -(1+\Vert {\varvec{\zeta }}\Vert ^2)+\zeta ^2_2 &{}\quad \cdots &{}\quad \zeta _{s-1} \zeta _2 \\ \vdots &{}\quad \vdots &{}\quad \ddots &{}\quad \vdots \\ \zeta _1 \zeta _{s-1} &{}\quad \zeta _2 \zeta _{s-1} &{}\quad \cdots &{}\quad -(1+\Vert {\varvec{\zeta }}\Vert ^2)+\zeta ^2_{s-1} \end{array}\right] . \end{aligned}$$

Define $\varvec{\psi }_{{\varvec{\zeta }}} = \left( {\varvec{\zeta }}^{\mathsf{T}}\ \varvec{\gamma }^{\mathsf{T}}\ \varvec{\beta }^{\mathsf{T}}\right) ^{\mathsf{T}},$ where $\varvec{\psi }_{{\varvec{\zeta }}}$ is one dimension lower than $\varvec{\psi }_{{\varvec{\theta }}} = \left( {\varvec{\theta }}^{\mathsf{T}}\ \varvec{\gamma }^{\mathsf{T}}\ \varvec{\beta }^{\mathsf{T}}\right) ^{\mathsf{T}}$. The Jacobian matrix of $\partial \varvec{\psi }/ \partial {{\varvec{\theta }}}$ of dimension $\dim ({\varvec{\psi }_{{\varvec{\theta }}}}) \times \big \{\dim (\varvec{\psi }_{{\varvec{\theta }}}) - 1\big \}$ is

$$\begin{aligned} \mathbf{J}({\varvec{\zeta }}) = \frac{\partial \varvec{\psi }_{{\varvec{\theta }}}}{\partial \varvec{\psi }_{{\varvec{\zeta }}}} = \left[ \begin{array}{cc} {{\varvec{\theta }}}^{(1)}({\varvec{\zeta }}) &{}\quad \mathbf{0}\\ \mathbf{0}&{}\quad \mathbf{I}\end{array}\right] . \end{aligned}$$

(8)

Note that this reparametrisation is preferred and in line with Yu and Ruppert (2002, p. 1046), where the norm of the reparametrised parameter strictly less than 1 is no longer assumed.