MSE dominance of the positive-part shrinkage estimator when each individual regression coefficient is estimated

Abstract

In this paper we consider a regression model and a general family of shrinkage estimators of regression coefficients. The estimation of each individual regression coefficient is important in some practical situations. Thus, we derive the formula for the mean squared error (MSE) of the general class of shrinkage estimators for each individual regression coefficient. It is shown analytically that the general family of shrinkage estimators is dominated by its positive-part variant in terms of MSE whenever there exists the positive-part variant or, in other words, the shrinkage factor can be negative for some parameter and data values.

Appendix

In this appendix, we derive the formulae for $H(p,q;c)$ and $J(p,q;c)$. First, we derive the formula for $H(p,q;c)$. Let $u_1=(h'\widehat{\gamma })^2/\sigma ^2$, $u_2=\widehat{\gamma }'(I_k -hh')\widehat{\gamma }/\sigma ^2$ and $u_3=e'e/\sigma ^2$. Then, $u_1 \sim \chi _1^{\prime 2}(\lambda _1)$ and $u_2 \sim \chi _{k-1}^{\prime 2}(\lambda _2)$, where $\chi _f^{\prime 2}(\lambda )$ is the noncentral chi-square distribution with $f$ degrees of freedom and noncentrality parameter $\lambda$, $\lambda _1=(h'\gamma )^2/\sigma ^2$ and $\lambda _2=\gamma '(I_k - hh')\gamma /\sigma ^2$. Further, $u_3$ is distributed as the chi-square distribution with $\nu =n-k$ degrees of freedom, and $u_1$, $u_2$ and $u_3$ are mutually independent.

Using $u_1$, $u_2$ and $u_3$, $H(p,q;c)$ is expressed as

$$\begin{aligned} H(p,q;c)&= (\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty } K_{ij} \int \!\!\!\int \!\!\!\int \limits _R \left( 1-\phi \left( \frac{u_1+u_2}{u_3}\right) \right) ^p \nonumber \\&\times u_1^{1/2+q+i-1}u_2^{(k-1)/2+j-1}u_3^{\nu /2-1} \exp [-(u_1\!+\!u_2\!+\!u_3)/2] du_1\, du_2 \, du_3,\nonumber \\ \end{aligned}$$

(25)

where

$$\begin{aligned} K_{ij}= \frac{w_i(\lambda _1)w_j(\lambda _2)}{2^{(\nu +k)/2+i+j}\Gamma (1/2+i)\Gamma ((k-1)/2+j)\Gamma (\nu /2)}, \end{aligned}$$

(26)

$w_i(\lambda )=\exp (-\lambda /2)(\lambda /2)^i/i!$, and $R$ is the region such that $(u_1+u_2)/u_3\ge c$.

Making use of the change of variables, $v_1=(u_1+u_2)/u_3$, $v_2=u_1 u_3/(u_1+u_2)$ and $v_3=u_3$, (25) reduces to

$$\begin{aligned}&(\sigma ^2)^q \sum _{i=0}^{\infty } \sum _{j=0}^{\infty } K_{ij} \int \limits _{0}^{\infty }\int \limits _{0}^{v_3} \int \limits _{c}^{\infty } \left( 1-\phi (v_1)\right) ^p v_1^{k/2+q+i+j-1} v_2^{1/2+q+i-1} v_3^{\nu /2} (v_3-v_2)^{(k-1)/2+j-1} \nonumber \\&\qquad \times \exp [-v_3(v_1+1)/2] dv_1\,dv_2\,dv_3. \end{aligned}$$

(27)

Again, making use of the change of variable, $z_1=v_2/v_3$, (27) reduces to

$$\begin{aligned}&(\sigma ^2)^q \sum _{i=0}^{\infty } \sum _{j=0}^{\infty } K_{ij} \frac{\Gamma (1/2+q+i)\Gamma ((k-1)/2+j)}{\Gamma (k/2+q+i+j)} \nonumber \\&\quad \times \int \limits _{0}^{\infty } \int \limits _{c}^{\infty } \left( 1\!-\!\phi (v_1)\right) ^p v_1^{k/2+q+i+j-1}v_3^{(\nu +k)/2+q+i+j-1} \exp [-v_3(v_1+1)/2] dv_1\, dv_3.\nonumber \\ \end{aligned}$$

(28)

Further, making use of the change of variable, $z_2=v_3(v_1+1)/2$, (28) reduces to

$$\begin{aligned}&(\sigma ^2)^q \sum _{i=0}^{\infty } \sum _{j=0}^{\infty } K_{ij} 2^{(\nu +k)/2+q+i+j} \frac{\Gamma (1/2\!+\!q\!+\!i)\Gamma ((k\!-\!1)/2\!+\!j)\Gamma ((\nu \!+\!k)/2\!+\!q\!+\!i\!+\!j)}{\Gamma (k/2\!+\!q\!+\!i\!+\!j)}\nonumber \\&\qquad \times \int \limits _{c}^{\infty } \left( 1-\phi (v_1)\right) ^p \frac{v_1^{k/2+q+i+j-1}}{(1+v_1)^{(\nu +k)/2+q+i+j}} dv_1. \end{aligned}$$

(29)

Finally, making use of the change of variable, $t=v_1/(1+v_1)$. we obtain (14) in the text.

Next, we derive the formula for $J(p,q;c)$. Differentiating $H(p,q;c)$ given in (14) with respect to $\gamma$, we have

$$\begin{aligned} \frac{\partial H(p,q;c)}{\partial \gamma }&= (2\sigma ^2)^q \sum _{i=0}^{\infty } \sum _{j=0}^{\infty } \left[ \frac{\partial w_i(\lambda _1)}{\partial \gamma }w_j(\lambda _2) +w_i(\lambda _1)\frac{\partial w_j(\lambda _2)}{\partial \gamma }\right] G_{ij}(p,q;c) \nonumber \\&= -\frac{hh'\gamma }{\sigma ^2}(2\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty }w_i(\lambda _1)w_j(\lambda _2) G_{ij}(p,q;c) \nonumber \\&+\frac{hh'\gamma }{\sigma ^2}(2\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty }w_i(\lambda _1)w_j(\lambda _2) G_{i+1,j}(p,q;c) \nonumber \\&-\frac{(I_k-hh')\gamma }{\sigma ^2}(2\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty }w_i(\lambda _1)w_j(\lambda _2) G_{ij}(p,q;c) \nonumber \\&+\frac{(I_k-hh')\gamma }{\sigma ^2}(2\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty }w_i(\lambda _1)w_j(\lambda _2) G_{i,j+1}(p,q;c), \end{aligned}$$

(30)

where we define $w_{-1}(\lambda _1) = w_{-1}(\lambda _2) =0$. Since $h'h=1$, we obtain

$$\begin{aligned} h'\frac{\partial H(p,q;c)}{\partial \gamma }&= -\frac{h'\gamma }{\sigma ^2}H(p,q;c)\nonumber \\&+\frac{h'\gamma }{\sigma ^2} (2\sigma ^2)^q \sum _{i=0}^{\infty }\sum _{j=0}^{\infty }w_i(\lambda _1)w_j(\lambda _2) G_{i+1,j}(p,q;c). \end{aligned}$$

(31)

Expressing $H(p,q;c)$ by $\widehat{\gamma }$ and $e'e$, we have

$$\begin{aligned} H(p,q;c)&= \int \!\!\!\int \limits _{F\ge c} \left( 1-\phi \left( F \right) \right) ^p (h'\widehat{\gamma })^{2q} f_N(\widehat{\gamma }) f_e(e'e) d\widehat{\gamma }\, d(e'e), \end{aligned}$$

(32)

where $F=(\widehat{\gamma }'\widehat{\gamma })/(e'e)$, $f(e'e)$ is the density function of $e'e$, and

$$\begin{aligned} f_N(\widehat{\gamma }) = \frac{1}{(2\pi )^{k/2}\sigma ^k} \exp \left[ -\frac{(\widehat{\gamma }-\gamma )'(\widehat{\gamma }-\gamma )}{2\sigma ^2}\right] , \end{aligned}$$

(33)

is the density function of $\widehat{\gamma }$.

Differentiating $H(p,q;c)$ given in (32) with respect to $\gamma$, and multiplying $h'$ from the left, we obtain

$$\begin{aligned} h'\frac{\partial H(p,q;c)}{\partial \gamma } \!=\! -\frac{h'\gamma }{\sigma ^2} H(p,q;c) \!+\!\frac{1}{\sigma ^2} E\left[ I(F\ge c) \left( 1-\phi \left( F \right) \right) ^p (h'\widehat{\gamma })^{2q}h'\widehat{\gamma } \right] .\nonumber \\ \end{aligned}$$

(34)

Equating (31) and (34), we obtain (15) in the text.