Semiparametric additive models under symmetric distributions

Abstract

In this paper we discuss estimation and diagnostic procedures in semiparametric additive models with symmetric errors in order to permit distributions with heavier and lighter tails than the normal ones, such as Student-t, Pearson VII, power exponential, logistics I and II, and contaminated normal, among others. Such models belong to the general class of statistical models GAMLSS proposed by Rigby and Stasinopoulos (Appl. Stat. 54:507–554, 2005). A back-fitting algorithm to attain the maximum penalized likelihood estimates (MPLEs) by using natural cubic smoothing splines is presented. In particular, the score functions and Fisher information matrices for the parameters of interest are expressed in a similar notation of that used in parametric symmetric models. Sufficient conditions on the existence of the MPLEs are presented as well as some inferential results and discussions on degrees of freedom and smoothing parameter estimation. Diagnostic quantities such as leverage, standardized residual and normal curvatures of local influence under two perturbation schemes are derived. A real data set previously analyzed under normal linear models is reanalyzed under semiparametric additive models with symmetric errors.

Appendices

Appendix A

Let D _a =diag_1≤i≤n (a _i ), with $a_{i}=- 2 (\zeta_{i} + 2 \zeta_{i}' \delta_{i} )$ and $\zeta _{i}'=\frac{\mathrm{d}\zeta_{i}}{\mathrm{d}\delta_{i}}$. Below we derive sufficient conditions to guarantee the concavity of the penalized log-likelihood function $L_{\mathrm{p}}(\mbox {\boldmath \(\beta\)\unboldmath }, \mathbf {f}_{1},\mathbf {f}_{2}, \phi, \mbox {\boldmath \(\alpha\)\unboldmath })$ in $\mbox {\boldmath \(\beta\)\unboldmath }$, f ₁, f ₂ and ϕ. In effect, we have the following.

(a′):

In the step (a) (Sect. 3.1), the concavity (in $\mbox {\boldmath \(\beta\)\unboldmath }$) of $L_{\mathrm{p}}(\mbox {\boldmath \(\beta\)\unboldmath }, \mathbf {f}_{1}, \mathbf {f}_{2}, \phi, \mbox {\boldmath \(\alpha\)\unboldmath })$ is guaranteed if only if the matrix $\mathbf {L}_{\mathrm{p}}^{\beta\beta} = -\frac{1}{\phi} \mathbf {X}^{T} \mathbf {D}_{a} \mathbf {X}\leq0$ (negative semidefinite) or, equivalently, if only if $- \mathbf {L}_{\mathrm{p}}^{\beta\beta}\geq0$ (non-negative definite). One has $- \mathbf {L}_{\mathrm{p}}^{\beta\beta}\geq0$ if the matrix D _a ≥0, that is, if a _i ≥0, ∀i=1,…,n.

(b′):

Then, in the step (b) (Sect. 3.1), one has concavity (in f ₁) of $L_{\mathrm{p}}^{c}(\mathbf {f}_{1}, \mathbf {f}_{2}, \phi, \mbox {\boldmath \(\alpha\)\unboldmath })$ if only if the matrix $\mathbf {L}_{\mathrm{p}}^{f_{1} f_{1}} = - ( \frac{1}{\phi} \mathbf {N}_{1}^{T} \mathbf {D}_{a} \mathbf {N}_{1} + \alpha_{1} \mathbf {K}_{1} ) \leq0$ or, equivalently, if only if $- \mathbf {L}_{\mathrm{p}}^{f_{1} f_{1}}\geq0$. Consequently, $- \mathbf {L}_{\mathrm{p}}^{f_{1} f_{1}} \geq 0$ if only if $\frac{1}{\phi} \mathbf {N}_{1}^{T} \mathbf {D}_{a} \mathbf {N}_{1}\geq0$ and α ₁ K ₁≥0. Since α ₁ is a positive scalar and K ₁≥0, we have α ₁ K ₁≥0. On the other hand, $\frac{1}{\phi} \mathbf {N}_{1}^{T} \mathbf {D}_{a} \mathbf {N}_{1}\geq 0$ if D _a ≥0, that is, if a _i ≥0, ∀i=1,…,n.

(c′):

Analogously, in the step (c) (Sect. 3.1), one has concavity (in f ₂) of $L_{\mathrm{p}}^{c}(\mathbf {f}_{2}, \phi, \mbox {\boldmath \(\alpha\)\unboldmath })$ if only if the matrix $\mathbf {L}_{\mathrm{p}}^{f_{2} f_{2}} = - ( \frac{1}{\phi} \mathbf {N}_{2}^{T} \mathbf {D}_{a} \mathbf {N}_{2} + \alpha_{2} \mathbf {K}_{2} )\leq0$ or, equivalently, if only if $- \mathbf {L}_{\mathrm{p}}^{f_{2} f_{2}}\geq0$. Consequently, $- \mathbf {L}_{\mathrm{p}}^{f_{2} f_{2}}\geq0$ if only if $\frac{1}{\phi} \mathbf {N}_{2}^{T} \mathbf {D}_{a} \mathbf {N}_{2}\geq 0$ and α ₂ K ₂≥0. Since α ₂ is a positive scalar and K ₂≥0, we have α ₂ K ₂≥0. On the other hand, $\frac{1}{\phi} \mathbf {N}_{2}^{T} \mathbf {D}_{a} \mathbf {N}_{2} \geq0$ if D _a ≥0, that is, if a _i ≥0, ∀i=1,…,n.

(d′):

Finally, in the step (d) (Sect. 3.1), the concavity (in ϕ) of $L_{\mathrm{p}}^{c}(\phi, \mbox {\boldmath \(\alpha\)\unboldmath })$ is guaranteed if only if $\partial^{2} L_{\mathrm{p}}^{c}(\phi, \mbox {\boldmath \(\alpha\)\unboldmath }) / \partial \phi^{2} < 0$, ∀ϕ.

Appendix B

2.1 B.1 Score function

Consider the penalized log-likelihood function given by (4). The score function of θ is given by $\mbox {\boldmath \(\mathrm{U}\)\unboldmath }_{\mathrm{p}} = \partial L_{\mathrm{p}}(\boldsymbol{\theta}, \mbox {\boldmath \(\alpha\)\unboldmath })/ \partial \mbox {\boldmath \(\theta\)\unboldmath }$. In particular, we obtain

where D _v is defined in Sect. 3.3 and $\mbox {\boldmath \(\epsilon \)\unboldmath }=\mathbf {y}-\mbox {\boldmath \(\mu\)\unboldmath }$, with $\mbox {\boldmath \(\mu\)\unboldmath }= \mathbf {X}\mbox {\boldmath \(\beta\)\unboldmath } + \sum_{k=1}^{s} \mathbf {N}_{k} \mathbf {f}_{k}$.

2.2 B.2 Hessian matrix

Let L _p (p ^∗×p ^∗) be the Hessian matrix with (j ^∗,ℓ ^∗)-element given by $\partial^{2} L_{\mathrm {p}}(\boldsymbol{\theta}, \mbox {\boldmath \(\alpha\)\unboldmath })/ \partial\theta_{j^{*}} \theta_{\ell^{*}}$, for j ^∗,ℓ ^∗=1,…,p ^∗. After some algebraic manipulations we find

where $\mathbf {D}_{\zeta'}=\mathrm{diag}_{1\leq i \leq n} (\zeta_{i}' )$, b=(b ₁,…,b _n )^T, $\mbox {\boldmath \(\delta\)\unboldmath }=(\delta_{1},\ldots,\delta_{n})^{T}$, and b _i =(ζ _i +ζ′ _i δ _i )ϵ _i , for i=1,…,n.

2.3 B.3 Expected information matrix

Let $\mathbf {D}_{d} =\frac{4 d_{g}}{\phi} \mathbf {I}_{(n,n)}$, with $d_{g}= \mathrm{E}(\zeta^{2}(\epsilon_{i}^{2}) \epsilon_{i}^{2})$, $f_{g}= \mathrm{E}(\zeta^{2}(\epsilon_{i}^{2}) \epsilon_{i}^{4})$ and ϵ _i ∼S(0,ϕ,g). By calculating the expectation of the −L _p we find that the (p ^∗×p ^∗) expected information matrix takes the following block-diagonal form:

$$\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p} = \left (\begin{array}{c@{\quad }c@{\quad }c@{\quad }c@{\quad }c@{\quad }c} \mathbf {X}^{T}\mathbf {D}_{d}\mathbf {X}& & \mathbf {X}^{T}\mathbf {D}_{d}\mathbf {N}_{1} & \ldots& \mathbf {X}^{T}\mathbf {D}_{d}\mathbf {N}_{s} & \mathbf {0}\\[3pt] \mathbf {N}_{1}^{T}\mathbf {D}_{d}\mathbf {X}&& \mathbf {N}_{1}^{T}\mathbf {D}_{d}\mathbf {N}_{1} + \alpha_{1} \mathbf {K}_{1} & \ldots& \mathbf {N}_{1}^{T}\mathbf {D}_{d}\mathbf {N}_{s} & \mathbf {0}\\[3pt] \vdots&& \vdots& \ddots& \vdots& \vdots\\[3pt] \mathbf {N}_{s}^{T}\mathbf {D}_{d}\mathbf {X}&& \mathbf {N}_{s}^{T}\mathbf {D}_{d}\mathbf {N}_{1}& \ldots& \mathbf {N}_{s}^{T}\mathbf {D}_{d}\mathbf {N}_{s} + \alpha_{s} \mathbf {K}_{s} & \vdots\\[3pt] \mathbf {0}&& \mathbf {0}& \ldots& \mathbf {0}& \mathcal{I}_\mathrm{p}^{\phi\phi} \end{array} \right ). $$

Now, taking $\mathcal{I}_{\mathrm{p}}^{\phi\phi}=\frac{n (4f_{g} - 1 )}{4\phi^{2}}$, $\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_{\mathrm{p}}^{\beta\beta} = \mathbf {X}^{T}\mathbf {D}_{d} \mathbf {X}$, $\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_{\mathrm{p}}^{\beta\mathrm{f}} = (\begin{array}{c@{\ }c@{\ }c} \mathbf {X}^{T}\mathbf {D}_{d} \mathbf {N}_{1} & \ldots& \mathbf {X}^{T}\mathbf {D}_{d} \mathbf {N}_{s} \\ \end{array} )$ and

$$\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\mathrm{f} \mathrm{f}} = \left (\begin{array}{c@{\quad }c@{\quad }c@{\quad }c} \mathbf {N}_{1}^{T}\mathbf {D}_{d} \mathbf {N}_{1} + \alpha_{1} \mathbf {K}_{1} & \ldots& \mathbf {N}_{1}^{T}\mathbf {D}_{d} \mathbf {N}_{s} \\[3pt] \vdots& \ddots& \vdots\\[3pt] \mathbf {N}_{s}^{T}\mathbf {D}_{d} \mathbf {N}_{1}& \ldots& \mathbf {N}_{s}^{T}\mathbf {D}_{d} \mathbf {N}_{s} + \alpha_{s} \mathbf {K}_{s} \end{array} \right ) , $$

we have

$$\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{-1}= \left (\begin{array}{c@{\quad }c@{\quad }c@{\quad }c} ( \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\beta} - \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\mathrm{f}} \mbox {\boldmath \(\mathcal {I}\)\unboldmath }_\mathrm{p}^{\mathrm{f}\mathrm{f}^{-1}} \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm {p}^{\beta\mathrm{f}^{T}} )^{-1} & \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\beta} \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\mathrm{f}} \mbox {\boldmath \(\mathcal {I}\)\unboldmath }_\mathrm{p}^{\mathrm{f} \mathrm{f}^{-1}} & \mathbf {0}\\[3pt] (\mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\beta} \mbox {\boldmath \(\mathcal {I}\)\unboldmath }_\mathrm{p}^{\beta\mathrm{f}} \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\mathrm{f} \mathrm{f}^{-1}} )^{T} & ( \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\mathrm{f} \mathrm{f}} - \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm{p}^{\beta\mathrm{f}^{T}} \mbox {\boldmath \(\mathcal {I}\)\unboldmath }_\mathrm{p}^{\beta\beta^{-1}} \mbox {\boldmath \(\mathcal{I}\)\unboldmath }_\mathrm {p}^{\beta\mathrm{f}} )^{-1} & \mathbf {0}\\[3pt] \mathbf {0}& \mathbf {0}& \mathcal{I}_\mathrm{p}^{\phi\phi^{-1}} \end{array} \right ). $$

Appendix C

In this appendix we present the expressions of the $\mbox {\boldmath \(\varDelta \)\unboldmath }_{\mathrm{p}} =\partial^{2} L_{\mathrm{p}}(\mbox {\boldmath \(\theta\)\unboldmath }, \mbox {\boldmath \(\alpha\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath }) / \partial \mbox {\boldmath \(\theta\)\unboldmath } \partial \mbox {\boldmath \(\omega\)\unboldmath }^{T}$ matrix for case-weight and explanatory variable perturbation schemes.

3.1 C.1 Cases-weight perturbation

Let us consider the attributed weights for the observations in the penalized log-likelihood function as $L_{\mathrm{p}}(\mbox {\boldmath \(\theta\)\unboldmath } , \mbox {\boldmath \(\alpha\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath })= L(\mbox {\boldmath \(\theta\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath }) - \sum_{k=1}^{s} \frac{\alpha_{k}}{2} \mathbf {f}_{k}^{T} \mathbf {K}_{k} \mathbf {f}_{k}$, where $L(\mbox {\boldmath \(\theta\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath })=\sum_{i=1}^{n} \omega_{i} L_{i}(\mbox {\boldmath \(\theta\)\unboldmath })$, $\mbox {\boldmath \(\omega\)\unboldmath }=(\omega_{1},\ldots,\omega_{n})^{T}$ is the vector of weights, with 0≤ω _i ≤1. In this case, the vector of no perturbation is given by $\mbox {\boldmath \(\omega\)\unboldmath }_{0}=\mathbf {1}_{(n \times 1)}$. Differentiating $L_{\mathrm{p}}(\mbox {\boldmath \(\theta\)\unboldmath } , \mbox {\boldmath \(\alpha\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath })$ with respect to the elements of $\mbox {\boldmath \(\theta\)\unboldmath }$ and ω _i , we obtain

3.2 C.2 Explanatory variable perturbation

Here the dth explanatory variable, assumed continuous, is perturbed by considering the additive perturbation scheme, namely x _idω =x _id +ω _i , where $\mbox {\boldmath \(\omega\)\unboldmath }=(\omega_{1},\ldots,\omega_{n})^{T}$ is the vector of perturbations such as $\omega_{i}\in\mathcal{R}$. In this case, the vector of no perturbation is given by $\mbox {\boldmath \(\omega\)\unboldmath }_{0}=\mathbf {0}_{(n \times1)}$ and the perturbed penalized log-likelihood function is given by $L_{\mathrm{p}}(\mbox {\boldmath \(\theta\)\unboldmath } , \mbox {\boldmath \(\alpha\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath })= L(\mbox {\boldmath \(\theta\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath }) - \sum_{k=1}^{s} \frac{\alpha_{k}}{2} \mathbf {f}_{k}^{T} \mathbf {K}_{k} \mathbf {f}_{k}$, where L(⋅) is given by (3) with δ _iω =ϕ ⁻¹(y _i −μ _iω )² in the place of δ _i and μ _iω =μ _i +ω _i β _d . Differentiating $L_{\mathrm{p}}(\mbox {\boldmath \(\theta\)\unboldmath } , \mbox {\boldmath \(\alpha\)\unboldmath } | \mbox {\boldmath \(\omega\)\unboldmath })$ with respect to the elements of $\mbox {\boldmath \(\theta\)\unboldmath }$ and ω _i , we obtain, after some algebraic manipulation,

Here $\mbox {\boldmath \(\mathrm{z}\)\unboldmath }_{d}$ denotes a (p×1) vector with 1 in the dth position and zero elsewhere, and $\widehat{\beta}_{d}$ denotes the dth element of $\widehat{\mbox {\boldmath \(\beta\)\unboldmath }}$.