Bayesian quantile regression for partially linear additive models

Abstract

In this article, we develop a semiparametric Bayesian estimation and model selection approach for partially linear additive models in conditional quantile regression. The asymmetric Laplace distribution provides a mechanism for Bayesian inferences of quantile regression models based on the check loss. The advantage of this new method is that nonlinear, linear and zero function components can be separated automatically and simultaneously during model fitting without the need of pre-specification or parameter tuning. This is achieved by spike-and-slab priors using two sets of indicator variables. For posterior inferences, we design an effective partially collapsed Gibbs sampler. Simulation studies are used to illustrate our algorithm. The proposed approach is further illustrated by applications to two real data sets.

Appendix: MCMC algorithm details

The joint distribution of all the variables is

$$\begin{aligned}&p(\varvec{\alpha }, \{\varvec{\beta }_j\}, \varvec{E},\mu ,\delta _{0}| \varvec{y}, \varvec{x})\\&\propto \exp \left\{ -\frac{1}{2}\left( \varvec{y}-\mu \varvec{1}_n-\varvec{B}_{0}\varvec{\alpha }-\sum \limits _{j=1}^{p}\varvec{B}_{j} \varvec{\beta }_{j}-k_{1}\varvec{e}\right) ^{T}\varvec{E}^{-1}\right. \\&\left. \left( \varvec{y}-\mu \varvec{1}_n-\varvec{B}_{0}\varvec{\alpha }-\sum \limits _{j=1}^{p}\varvec{B}_{j} \varvec{\beta }_{j}-k_{1}\varvec{e}\right) \right\} \times \det [\varvec{E}]^{-1/2}\\&\quad \times p(\varvec{\alpha })\times \prod \limits _{j=1}^{p}p(\varvec{\beta }_j)\times p(\delta _{0})\times \prod \limits _{i=1}^{n}p(e_i)\times p(\mu ), \end{aligned}$$

where $p(\varvec{\beta }_j)$, $p(\varvec{\alpha })$, $p(e_i)$, $p(\delta _{0})$ and $p(\mu )$ are the prior distributions of $\varvec{\beta }_j$, $\varvec{\alpha }$, $e_i$, $\delta _{0}$, and $\mu$ respectively.

We use the Metropolis-within-Gibbs algorithm to sample from the posterior distribution. We integrate out $\alpha _j$ in step 3 and $\varvec{\beta }_j$ in step 5 to improve mixing of the Markov Chain. The posterior distribution of each variable is as follows ($\sim$ denotes all variables except the one to be sampled):

• 1. Sample $p(\alpha _j|\sim )=p \Big (\alpha _j|\varvec{y}^{*},\varvec{e}, \delta _{0}, \sigma _j^{2},\gamma ^{(\alpha )}_j \Big ),~ j=1, \ldots , p$, from the conditional distribution of $\alpha _j$,

  $$\begin{aligned} \begin{aligned} p\Big (\alpha _j|\varvec{y}^{*},\varvec{e},\delta _{0}, \sigma _j^{2},\gamma ^{(\alpha )}_j=1\Big )&\sim N (\mu _j, \xi _j^{2}),\\ p\Big (\alpha _j|\varvec{y}^{*},\varvec{e},\delta _{0}, \sigma _j^{2},\gamma ^{(\alpha )}_j=0 \Big )&=0, \end{aligned} \end{aligned}$$

  where $\varvec{y}^{*}=\varvec{y}-\mu \varvec{1}_n-\sum \limits _{i\ne j}^{p} \alpha _i\varvec{B}_{i0}-\sum \limits _{i=1 }^{p} \varvec{B}_{i} \varvec{\beta }_{i}-k_{1}\varvec{e}$, $\xi ^{2}_j =\Big (\varvec{B}^{T}_{j0}\varvec{E}^{-1}\varvec{B}_{j0}+\frac{1}{\sigma _j^{2}}\Big )^{-1}$ and $\mu _j=\xi ^{2}_j \varvec{B}^{T}_{j0} \varvec{E}^{-1}\varvec{y}^{*}$.

• 2. Sample $p(\mu |\sim )=p (\mu |\varvec{y}^{*},\varvec{e},\delta _{0})$, from the conditional distribution of $\mu$,

  $$\begin{aligned} \begin{aligned} p(\mu |\varvec{y}^{*},\varvec{e}, \delta _{0})&\sim N (\mu _0, \xi _0^{2}), \end{aligned} \end{aligned}$$

  where $\xi ^{2}_0=k_2\delta _{0}(\sum \nolimits _{i=1}^{n}e_i^{-1})^{-1}$, $\mu _0=\xi ^{2}_0\varvec{1}^{T}_{n} \varvec{E}^{-1}\varvec{y}^{*}$, and $\varvec{y}^{*}=\varvec{y}-\sum \nolimits _{i=1}^{p} \alpha _i\varvec{B}_{i0}-\sum \nolimits _{i=1 }^{p} \varvec{B}_{i} \varvec{\beta }_{i}-k_1\varvec{e}$.

• 3. Sample $p(\gamma ^{(\alpha )}_j|\sim )=p(\gamma ^{(\alpha )}_j|\varvec{y}^{*}, \varvec{e},\delta _{0}, \sigma _j^{2}), ~j=1, \ldots , p$, from its conditional posterior after integrating over $\alpha _j$, $p(\gamma ^{(\alpha )}_j=1|\varvec{y}^{*}, \varvec{e},\delta _{0}, \sigma _j^{2})=\frac{1}{1+h}$, with

  $$\begin{aligned} \begin{aligned} h~=&~h_1\frac{p\Big (\gamma _j^{(\alpha )}=0|\gamma ^{(\alpha )}_{i\ne j}\Big )}{p\Big (\gamma _j^{(\alpha )}=1|\gamma ^{(\alpha )}_{i\ne j}\Big )},\\ h_1=&~\exp \Big \{-\frac{1}{2}(\varvec{B}^{T}_{j0}\varvec{E}^{-1} \varvec{y}^{*})^{2}\Big (\varvec{B}^{T}_{j0}\varvec{E}^{-1} \varvec{B}_{j0}\!+\!\frac{1}{\sigma ^2_j}\Big )^{-1}\Big \}\\&\times (\sigma _j^{2}\varvec{B}^{T}_{j0}\varvec{E}^{-1} \varvec{B}_{j0}+1)^{\frac{1}{2}},\\ \end{aligned} \end{aligned}$$

  where $p(\gamma ^{(\alpha )}_j=0|\gamma ^{(\alpha )}_{i\ne j})=(p-q_{\gamma ^{(\alpha )}_{0}})/(p+1)$ and $p(\gamma ^{(\alpha )}_j=1|\gamma ^{(\alpha )}_{i\ne j})=(1+q_{\gamma ^{(\alpha )}_{0}})/(p+1)$ with $\gamma ^{(\alpha )}_{0}=(\gamma ^{(\alpha )}_1,\ldots ,\gamma ^{(\alpha )}_{j-1},0,\gamma ^{(\alpha )}_{j+1},\ldots ,\gamma ^{(\alpha )}_{p})^{T}$, and $\varvec{y}^{*}=\varvec{y}-\mu \varvec{1}_n-\sum \nolimits _{i\ne j}^{p} \alpha _i\varvec{B}_{i0}-\sum \nolimits _{i=1 }^{p} \varvec{B}_{i} \varvec{\beta }_{i}-k_{1}\varvec{e}$.

• 4. Sample $p(\varvec{\beta }_j|\sim )=p(\varvec{\beta }_j|\varvec{y}^{*}, \varvec{e}, \delta _{0}, \tau ^{2}_j, \gamma ^{(\varvec{\beta })}_j), ~j=1, \ldots , p$,

  $$\begin{aligned} \begin{aligned} p\Big (\varvec{\beta }_j|\varvec{y}^{*},\varvec{e},\delta _{0}, \tau _j^{2},\gamma ^{(\varvec{\beta })}_j=1\Big )&\sim N (\varvec{\mu }_j, \varvec{\Sigma }_j),\\ p\Big (\varvec{\beta }_j|\varvec{y}^{*},\varvec{e},\delta _{0}, \tau _j^{2},\gamma ^{(\varvec{\beta })}_j=0\Big )&=0, \end{aligned} \end{aligned}$$

  where $\varvec{y}^{*}=\varvec{y}-\mu \varvec{1}_n-\sum \nolimits _{i=1 }^{p} \alpha _i \varvec{B}_{i0}-\sum \nolimits _{i\ne j }^{p} \varvec{B}_{i} \varvec{\beta }_{i}-k_1\varvec{e}$, $\varvec{\mu }_j=\varvec{\Sigma }_j \varvec{B}^{T}_j \varvec{E}^{-1}\varvec{y}^{*}$, $\varvec{\Sigma }_j=(\varvec{B}^{T}_j \varvec{E}^{-1} \varvec{B}_j+\frac{1}{\tau ^{2}_j}\varvec{\Omega }_j)^{-1}$.

• 5. Sample $p(\gamma ^{(\varvec{\beta })}_j|\sim )=p(\gamma ^{(\varvec{\beta })}_j|\varvec{y}^{*},\varvec{e},\delta _{0}, \tau _j^{2}),~ j=1, \ldots , p$, from its conditional posterior after integrating over $\varvec{\beta }_j$, $p(\gamma ^{(\varvec{\beta })}_j=1|\varvec{y}^{*},\varvec{e},\delta _{0}, \tau _j^{2})=\frac{1}{1+h}$, with

  $$\begin{aligned} h&= h_{1}\frac{p\Big (\gamma _j^{(\varvec{\beta })}=0|~\gamma ^{(\varvec{\beta })}_{i\ne j}\Big )}{p\Big (\gamma _j^{(\varvec{\beta })}=1|~\gamma ^{(\varvec{\beta })}_{i\ne j}\Big )},\\ h_1&= \exp \Big \{-\frac{1}{2}\Big [\varvec{y}^{*T}\varvec{E}^{-1}\varvec{B}_j\Big (\varvec{B}_j^{T}\varvec{E}^{-1}\varvec{B}_j\\&+\frac{1}{\tau _j^{2}}\varvec{\Omega }_j\Big )^{-1}\varvec{B}_j^T \varvec{E}^{-1}\varvec{y}^{*}\Big ]\Big \}\\&\times \det \Big [\frac{1}{\tau ^{2}_{j}}\varvec{\Omega }_j\Big ]^{-\frac{1}{2}}\times \det \Big [\varvec{B}_j^T\varvec{E}^{-1} \varvec{B}_j+\frac{1}{\tau _j^{2}}\varvec{\Omega }_j\Big ]^{\frac{1}{2}}, \end{aligned}$$

  where $\varvec{y}^{*}=\varvec{y}-\mu \varvec{1}_n-\sum \nolimits _{i=1 }^{p} \alpha _i \varvec{B}_{i0}-\sum \nolimits _{i\ne j }^{p} \varvec{B}_{i} \varvec{\beta }_{i}-k_1\varvec{e}$, $p(\gamma ^{(\varvec{\beta })}_j=0|\gamma ^{(\varvec{\beta })}_{i\ne j})=(p-q_{\gamma ^{(\varvec{\beta })}_{0}})/(p+1)$ and $p(\gamma ^{(\varvec{\beta })}_j=1|\gamma ^{(\varvec{\beta })}_{i\ne j})=(1+q_{\gamma ^{(\varvec{\beta })}_{0}})/(p+1)$ with $\gamma ^{(\varvec{\beta })}_{0}=(\gamma ^{(\varvec{\beta })}_1,\ldots ,\gamma ^{(\varvec{\beta })}_{j-1},0,\gamma ^{(\varvec{\beta })}_{j+1},\ldots ,\gamma ^{(\varvec{\beta })}_{p})^{T}$.

• 6. Sample $\delta _0$,

  $$\begin{aligned} \begin{aligned} \delta _{0}&\sim IG (a_1+3n/2, \nu ),\\ \nu&=a_2+\left( (2k_2e_i)^{-1}\sum \limits _{i=1}^{n}\left( y_{i}-\sum \limits _{j=1}^{p}\alpha _{j} B_{j0}(x_{ij})\right. \right. \\&\quad \left. \left. -\sum \limits _{j=1}^{p}\sum \limits _{k=1}^{K}\beta _{jk}B_{jk}(x_{ij})-k_{1}e_{i}\right) ^{2}+e_{i}\right) .\\ \end{aligned} \end{aligned}$$

• 7. Sample $\sigma ^{2}_j,~ j=1,\ldots ,p$, and $\tau ^{2}_j, ~j=1,\ldots ,p$, from their conditional posterior distributions if $\gamma ^{(\alpha )}_j$, $\gamma ^{(\varvec{\beta })}_{j}\ne 0$,

  $$\begin{aligned} \begin{aligned} \sigma ^{2}_j&\sim IG (a_1+1/2, a_2+(\alpha _j^{2}/2)),\\ \tau ^2_j&\sim IG(a_1+K /2, a_2+(\varvec{\beta }_j^{T} \varvec{\Omega }_j\varvec{\beta }_j/2)).\\ \end{aligned} \end{aligned}$$

  Otherwise they are generated from their priors.

• 8. The full conditional distribution of $e_{i}, ~i=1,\ldots , n$ is a generalized inverse Gaussian distribution $(\textit{GIG})$,

  $$\begin{aligned}&p(e_{i}|\delta _0,\nu _i)\\&\quad \sim \textit{GIG}\left( \frac{1}{2}, \sqrt{\frac{(y_{i}-\nu _i)^{2}}{k_{2}\delta _0}},\sqrt{\frac{k_{1}^{2}}{k_{2}\delta _0}+\frac{2}{\delta _0}}\right) ,\\&\nu _i=y_{i}-\mu -\sum \limits _{j=1}^{p}\alpha _{j} B_{j0}(x_{ij})-\sum \limits _{j=1}^{p}\sum \limits _{k=1}^{K}\beta _{jk}B_{jk}(x_{ij}), \end{aligned}$$

  where the probability density function of $\textit{GIG}(\rho ,m,n)$ is

  $$\begin{aligned}&f(x|\rho ,m,n)\\&\quad =\frac{(n/m)^{\rho }}{2K_{\rho }(mn)}x^{\rho -1}\exp \Big \{-\frac{1}{2}(m^{2}x^{-1}+n^{2}x)\Big \},\\&\qquad x>0,\; -\infty <\rho <\infty ,\; m\ge 0,\; n\ge 0, \end{aligned}$$

  and $K_{\rho }$ is the modified Bessel function of the third kind (Barndorff-Nielsen and Shephard 2001).