Skip to main content

A Robust Bayesian Analysis of Variable Selection under Prior Ignorance

Abstract

We propose a cautious Bayesian variable selection routine by investigating the sensitivity of a hierarchical model, where the regression coefficients are specified by spike and slab priors. We exploit the use of latent variables to understand the importance of the co-variates. These latent variables also allow us to obtain the size of the model space which is an important aspect of high dimensional problems. In our approach, instead of fixing a single prior, we adopt a specific type of robust Bayesian analysis, where we consider a set of priors within the same parametric family to specify the selection probabilities of these latent variables. We achieve that by considering a set of expected prior selection probabilities, which allows us to perform a sensitivity analysis to understand the effect of prior elicitation on the variable selection. The sensitivity analysis provides us sets of posteriors for the regression coefficients as well as the selection indicators and we show that the posterior odds of the model selection probabilities are monotone with respect to the prior expectations of the selection probabilities. We also analyse synthetic and real life datasets to illustrate our cautious variable selection method and compare it with other well known methods.

This is a preview of subscription content, access via your institution.

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

References

Download references

Acknowledgments

This work is partially funded by the European Commission’s H2020 programme, through the UTOPIAE Marie Curie Innovative Training Network, H2020-MSCA-ITN-2016, Grant Agreement number 722734.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tathagata Basu.

Ethics declarations

Conflict of Interest

Authors declare that they have no conflict of interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix: Orthogonal Design Case

Lemma A.1.

Let,

$$ {\upbeta}_{j} \mid y \sim\frac{\alpha_{j}w_{1,j}}{W_{j}} \mathcal{N}\left( \hat{\upbeta}_{1,j}, {\sigma_{1}^{2}}\right) +\frac{(1-\alpha_{j})w_{0,j}}{W_{j}} \mathcal{N}\left( \hat{\upbeta}_{0,j}, {\sigma_{0}^{2}}\right). $$
(1)

Then, \(\mathcal {N}\left (\hat {\upbeta }_{1,j}, {\sigma _{1}^{2}}\right )\) dominates the posterior if

$$ \begin{array}{@{}rcl@{}} \hat{{{\upbeta}_{j}^{2}}} > \frac{\sigma^{2}}{n}\frac{(n{\tau_{1}^{2}}+1)(n{\tau_{0}^{2}}+1)}{n{\tau_{1}^{2}}-n{\tau_{0}^{2}}} \left[ 2\ln\left( \frac{1-\epsilon_{1}}{\epsilon_{1}}\right) +\ln\left( \frac{n{\tau_{1}^{2}}+1}{n{\tau_{0}^{2}}+1}\right)\right] \end{array} $$
(2)

and \(\mathcal {N}\left (\hat {\upbeta }_{0,j}, {\sigma _{0}^{2}}\right )\) dominates the posterior if,

$$ \begin{array}{@{}rcl@{}} \hat{{{\upbeta}_{j}^{2}}} < \frac{\sigma^{2}}{n}\frac{(n{\tau_{1}^{2}}+1)(n{\tau_{0}^{2}}+1)}{n{\tau_{1}^{2}}-n{\tau_{0}^{2}}} \left[ 2\ln\left( \frac{\epsilon_{2}}{1-\epsilon_{2}}\right) +\ln\left( \frac{n{\tau_{1}^{2}}+1}{n{\tau_{0}^{2}}+1}\right)\right]. \end{array} $$
(3)

Proof 4.

Exploiting the monotonicity property of the posterior odds, we can say that \(\mathcal {N}\left (\hat {\upbeta }_{1,j}, {\sigma _{1}^{2}}\right )\) dominates the posterior if \(\frac {\epsilon _{1}}{(1-\epsilon _{1})}\cdot \frac {w_{1,j}}{w_{0,j}}>1\). That is, if

$$ \begin{array}{@{}rcl@{}} \exp\left( -\frac{n\hat{{{\upbeta}_{j}^{2}}}}{2(n\sigma^{2}{\tau_{1}^{2}}+\sigma^{2})} + \frac{n\hat{{{\upbeta}_{j}^{2}}}}{2(n\sigma^{2}{\tau_{0}^{2}}+\sigma^{2})}\right) \!\!\!\!\!\!\!\!&&>\!\frac{(1 - \epsilon_{1})\sqrt{n{\tau_{1}^{2}}+1}}{\epsilon_{1}\sqrt{n{\tau_{0}^{2}}+1}} \end{array} $$
(4)
$$ \begin{array}{@{}rcl@{}} -\frac{n\hat{{{\upbeta}_{j}^{2}}}}{2(n\sigma^{2}{\tau_{1}^{2}}+\sigma^{2})} +\frac{n\hat{{{\upbeta}_{j}^{2}}}}{2(n\sigma^{2}{\tau_{0}^{2}}+\sigma^{2})} \!\!\!\!\!\!\!\!&&>\!\ln\left( \frac{(1 - \epsilon_{1})\sqrt{n{\tau_{1}^{2}}+1}}{\epsilon_{1}\sqrt{n{\tau_{0}^{2}}+1}}\right) \end{array} $$
(5)
$$ \begin{array}{@{}rcl@{}} \frac{n\hat{{{\upbeta}_{j}^{2}}}}{2\sigma^{2}} \left[-\frac{1}{(n{\tau_{1}^{2}}+1)}+\frac{1}{(n{\tau_{0}^{2}}+1)}\right] \!\!\!\!\!\!\!\!&&>\!\ln\left( \frac{(1 - \epsilon_{1})\sqrt{n{\tau_{1}^{2}}+1}}{\epsilon_{1}\sqrt{n{\tau_{0}^{2}}+1}}\right) \end{array} $$
(6)
$$ \begin{array}{@{}rcl@{}} \frac{n\hat{{{\upbeta}_{j}^{2}}}}{2\sigma^{2}} \frac{n{\tau_{1}^{2}}-n{\tau_{0}^{2}}}{(n{\tau_{1}^{2}}+1)(n{\tau_{0}^{2}}+1)} \!\!\!\!\!\!\!\!&&>\!\ln\left( \frac{(1 - \epsilon_{1})\sqrt{n{\tau_{1}^{2}}+1}}{\epsilon_{1}\sqrt{n{\tau_{0}^{2}}+1}}\right).~~~~~~~~~ \end{array} $$
(7)

Then after rearranging the terms on both sides, we get:

$$ \begin{array}{@{}rcl@{}} \hat{{{\upbeta}_{j}^{2}}} > \frac{\sigma^{2}}{n}\frac{(n{\tau_{1}^{2}}+1)(n{\tau_{0}^{2}}+1)}{n{\tau_{1}^{2}}-n{\tau_{0}^{2}}} \left[ 2\ln\left( \frac{1-\epsilon_{1}}{\epsilon_{1}}\right) +\ln\left( \frac{n{\tau_{1}^{2}}+1}{n{\tau_{0}^{2}}+1}\right)\right]. \end{array} $$
(8)

Similarly we can prove the other inequality. □

Lemma A.2.

Let \(g:(0,1)\to \mathbb {R}\) be defined as

$$ g(\alpha) := \frac{a\alpha + b}{c\alpha + d} $$
(9)

for some constants a, b, c, and \(d\in \mathbb {R}\) such that cα + d > 0 ∀α ∈ (0,1). Then g is monotonically increasing when adbc > 0 and monotonically decreasing when adbc < 0.

Proof 5.

The proof is straightforward and therefore we omit it. □

Note that, the posterior mean of βj can be written as:

$$ \begin{array}{@{}rcl@{}} E({\upbeta}_{j}\mid y) = \frac{(w_{1,j}\hat{\upbeta}_{1,j} - w_{0,j}\hat{\upbeta}_{1,j})\alpha_{j} + w_{0,j}\hat{\upbeta}_{0,j}} {(w_{1,j}-w_{0,j})\alpha_{j} + w_{0,j}}. \end{array} $$
(10)

Therefore, we have

$$ \begin{array}{@{}rcl@{}} \frac{d}{d\alpha_{j}}E({\upbeta}_{j}\mid y) \!\!\!\!\!\!\!\!\!&&= \frac{w_{0, j}w_{1, j} (\hat{\upbeta}_{1,j}-\hat{\upbeta}_{0,j})}{[(w_{1,j}-w_{0,j})\alpha_{j} + w_{0,j}]^{2}} \end{array} $$
(11)
$$ \begin{array}{@{}rcl@{}} &&= \frac{w_{0, j}w_{1, j}} {[(w_{1,j}-w_{0,j})\alpha_{j} + w_{0,j}]^{2}} \left( \frac{n{\tau_{1}^{2}}\hat{{\upbeta}_{j}}}{n{\tau_{1}^{2}} +1} -\frac{n{\tau_{0}^{2}}\hat{{\upbeta}_{j}}}{n{\tau_{0}^{2}} +1}\right) \end{array} $$
(12)
$$ \begin{array}{@{}rcl@{}} &&= \frac{w_{0, j}w_{1, j}\hat{\upbeta}_{j}} {[(w_{1,j}-w_{0,j})\alpha_{j} + w_{0,j}]^{2}} \left( \frac{n{\tau_{1}^{2}}}{n{\tau_{1}^{2}} +1} - \frac{n{\tau_{0}^{2}}}{n{\tau_{0}^{2}} +1}\right) \end{array} $$
(13)
$$ \begin{array}{@{}rcl@{}} &&= \frac{w_{0, j}w_{1, j}} {[(w_{1,j}-w_{0,j})\alpha_{j} + w_{0,j}]^{2}} \frac{n{\tau_{1}^{2}} - n{\tau_{0}^{2}}}{(n{\tau_{1}^{2}} +1)(n{\tau_{0}^{2}} +1)}\hat{\upbeta}_{j}. ~~~~~~~~~~~~~~\end{array} $$
(14)

Since τ1 > τ0, the posterior mean of βj is monotonically increasing with respect to αj for \(\hat {\upbeta }_{j} > 0\) and monotonically decreasing with respect to αj for \(\hat {\upbeta }_{j} < 0\).

Lemma A.3.

Let

$$ X\sim w_{1} f_{1} + w_{2} f_{2} $$
(15)

where fi denotes a normal density with mean μi and variance \({\sigma ^{2}_{i}}\) for i = 1,2. Then,

$$ Var(X) = {\sum}_{i=1}^{2} w_{i}({\sigma^{2}_{i}} + {\mu_{i}^{2}}) - \left( {\sum}_{i=1}^{2} w_{i}\mu_{i}\right)^{2}. $$
(16)

Proof 6.

First, note that

$$ \begin{array}{@{}rcl@{}} E(X^{2}){} &&=\int x^{2}[w_{1}f_{1}(x) + w_{2} f_{2}(x)]dx \end{array} $$
(17)
$$ \begin{array}{@{}rcl@{}} && = w_{1}\int x^{2}f_{1}(x)dx + w_{2}\int x^{2}f_{2}(x)dx \end{array} $$
(18)
$$ \begin{array}{@{}rcl@{}} && = w_{1}({\sigma^{2}_{1}} + {\mu_{1}^{2}}) + w_{2}({\sigma^{2}_{2}} + {\mu_{2}^{2}}). \end{array} $$
(19)

Consequently, Then, the variance of X is given by:

$$ \begin{array}{@{}rcl@{}} \text{Var}(X) &&= E(X^{2}) - \left[E(X)\right]^{2} \end{array} $$
(20)
$$ \begin{array}{@{}rcl@{}} && = {\sum}_{i=1}^{2}w_{i}({\sigma^{2}_{i}} + {\mu_{i}^{2}}) - \left( {\sum}_{i=1}^{2} w_{i}\mu_{i}\right)^{2}. \end{array} $$
(21)

Now, we know that,

$$ \begin{array}{@{}rcl@{}} {\upbeta}_{j} \mid y \sim\frac{\alpha_{j}w_{1,j}}{W_{j}} \mathcal{N}\left( \hat{\upbeta}_{1,j}, {\sigma_{1}^{2}}\right) +\frac{(1-\alpha_{j})w_{0,j}}{W_{j}} \mathcal{N}\left( \hat{\upbeta}_{0,j}, {\sigma_{0}^{2}}\right). \end{array} $$
(22)

Then from above lemma, we can show that the variance of βjy is given by:

$$ \begin{array}{@{}rcl@{}} \text{Var}({\upbeta}_{j}\mid y) &=& \frac{\alpha_{j}w_{1,j}{\sigma^{2}_{1}} + (1-\alpha_{j})w_{0,j}{\sigma^{2}_{0}}}{W_{j}}\\ &&+ \frac{\alpha(1-\alpha)w_{1,j}w_{0,j}(\hat{\upbeta}_{1,j} - \hat{\upbeta}_{0,j})^{2}} {{W_{j}^{2}}}. \end{array} $$
(23)

Appendix: Regression Coefficients

$$ \begin{array}{@{}rcl@{}} P(\upbeta\mid y) \!\!\!\!\!\!\!\!\!&&\overset{\upbeta}{\propto}\int \int {\sum}_{\gamma} P(\upbeta, \gamma, \sigma^{2}, q \mid y) dq d\sigma^{2} \end{array} $$
(1)
$$ \begin{array}{@{}rcl@{}} &&\overset{\upbeta}{\propto}\int P(y\mid \upbeta, \sigma^{2}) {\sum}_{\gamma}P(\upbeta \mid \gamma, \sigma^{2}) \left[\int P(\gamma\mid q) P(q) dq\right]\\&& P(\sigma^{2}) d \sigma^{2} \end{array} $$
(2)
$$ \begin{array}{@{}rcl@{}} &&\overset{\upbeta}{\propto}\int P(y\mid \upbeta, \sigma^{2}) {\prod}_{j}\left[\alpha_{j}f_{1}({\upbeta}_{j})+ (1-\alpha_{j})f_{0}({\upbeta}_{j})\right] P(\sigma^{2}) d \sigma^{2}.~~~~~~~~~~~ \end{array} $$
(3)

To simplify the product term in the above expression, we first provide the following identity:

Lemma B.1.

Let,

$$ f_{\gamma_{j}}({\upbeta}_{j}) := \frac{1}{\sqrt{2\pi\sigma^{2}\tau_{1}^{2\gamma_{j}}\tau_{0}^{2(1-\gamma_{j})}}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}\tau_{1}^{2\gamma_{j}}\tau_{0}^{2(1-\gamma_{j})}}\right). $$
(4)

Then,

$$ \begin{array}{@{}rcl@{}} &&{\prod}_{j}\left[\alpha_{j}f_{1}({\upbeta}_{j})+ (1-\alpha_{j})f_{0}({\upbeta}_{j})\right]\\ &&= \frac{1}{\sqrt{(2\pi\sigma^{2}{\tau_{1}^{2}})^{p}}}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}{\tau_{1}^{2}}}\right) \left( {\prod}_{j}\alpha_{j}\right)\left( {\sum}_{k=0}^{p}g_{k}(\upbeta, \sigma^{2})\right) , \end{array} $$
(5)

where

$$ \begin{array}{@{}rcl@{}} &&g_{0}(\upbeta, \sigma^{2}) = 1, \end{array} $$
(6)
$$ \begin{array}{@{}rcl@{}} &&g_{p}(\upbeta, \sigma^{2}) = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}} \left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right) {\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}}, \end{array} $$
(7)
$$ \begin{array}{@{}rcl@{}} &&\text{and for }\ 1\le k \le p-1,\\ &&g_{k}(\upbeta, \sigma^{2}) = \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}}\exp\\ &&\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}+\cdots+{\upbeta}_{j_{k}}^{2}}{2\sigma^{2}} \left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right). \end{array} $$
(8)

Proof 7.

From the left hand side, we have

$$ \begin{array}{@{}rcl@{}} &&{\prod}_{j}\left[\alpha_{j}f_{1}({\upbeta}_{j})+ (1-\alpha_{j})f_{0}({\upbeta}_{j})\right]\\ &&={\prod}_{j}\alpha_{j}f_{1}({\upbeta}_{j}){\prod}_{j}\left[1+ \frac{1-\alpha_{j}}{\alpha_{j}} \frac{f_{0}({\upbeta}_{j})}{f_{1}({\upbeta}_{j})}\right] \end{array} $$
(9)
$$ \begin{array}{@{}rcl@{}} &&={\prod}_{j}\frac{\alpha_{j}}{\sqrt{2\pi\sigma^{2}{\tau_{1}^{2}}}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}{\tau_{1}^{2}}}\right){\prod}_{j}\\ &&\left[1+ \frac{\tau_{1}(1-\alpha_{j})}{\tau_{0}\alpha_{j}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\right] \end{array} $$
(10)
$$ \begin{array}{@{}rcl@{}} &&=\frac{1}{\sqrt{(2\pi\sigma^{2}{\tau_{1}^{2}})^{p}}}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}{\tau_{1}^{2}}}\right) {\prod}_{j}\alpha_{j}{\prod}_{j}\\ && \left[1+ \frac{\tau_{1}(1-\alpha_{j})}{\tau_{0}\alpha_{j}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\right]. \end{array} $$
(11)

Now to evaluate the product term \({\prod }_{j} \left [1+ \frac {\tau _{1}(1-\alpha _{j})}{\tau _{0}\alpha _{j}} \exp \left (-\frac {{{\upbeta }_{j}^{2}}}{2\sigma ^{2}}\left (\frac {1}{{\tau _{0}^{2}}} -\frac {1}{{\tau _{1}^{2}}}\right )\right )\right ]\), we use the following identity

$$ \begin{array}{@{}rcl@{}} &&{\prod}_{j}(1+a_{j}) \\ &&= 1 + {\sum}_{j}a_{j} + {\sum}_{j_{1}<j_{2}}a_{j_{1}}a_{j_{2}} + {\sum}_{j_{1}<j_{2}<j_{3}}a_{j_{1}}a_{j_{2}}a_{j_{3}}\\ &&+ {\cdots} + {\sum}_{j_{1}<\cdots<j_{(p-1)}}(a_{j_{1}} {\cdots} a_{j_{(p-1)}})+ {\prod}_{j}a_{j}. \end{array} $$
(12)

Then, combining eq. 11 and eq. 12, we have

$$ \begin{array}{@{}rcl@{}} &&{\prod}_{j} \left[1+ \frac{\tau_{1}(1-\alpha_{j})}{\tau_{0}\alpha_{j}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\right]\\ &&=1 + \frac{\tau_{1}}{\tau_{0}}{\sum}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \exp\left( -\frac{{{\upbeta}_{j}^{2}}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\\ && + \frac{{\tau_{1}^{2}}}{{\tau_{0}^{2}}} {\sum}_{j_{1}<j_{2}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \exp\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\\ && + \frac{{\tau_{1}^{3}}}{{\tau_{0}^{3}}} {\sum}_{j_{1}<j_{2}<j_{3}}\frac{1 - \alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1 - \alpha_{j_{2}}}{\alpha_{j_{2}}} \frac{1 - \alpha_{j_{3}}}{\alpha_{j_{3}}} \exp\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}+{\upbeta}_{j_{3}}^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} - \frac{1}{{\tau_{1}^{2}}}\right)\right)\\ && + \cdots\\ && + \frac{\tau_{1}^{(p-1)}}{\tau_{0}^{(p-1)}} {\sum}_{j_{1}<\cdots<j_{p-1}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{(p-1)}}}{\alpha_{j_{(p-1)}}}\\ &&\exp\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}+\cdots+{\upbeta}_{j_{(p-1)}}^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)\\ && + \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right) {\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}}. \end{array} $$
(13)

Therefore,

$$ \begin{array}{@{}rcl@{}} &&{\prod}_{j}\left[\alpha_{j}f_{1}({\upbeta}_{j})+ (1-\alpha_{j})f_{0}({\upbeta}_{j})\right]\\ &&= \frac{1}{\sqrt{(2\pi\sigma^{2}{\tau_{1}^{2}})^{p}}}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}{\tau_{1}^{2}}}\right) \left( {\prod}_{j}\alpha_{j}\right)\left( {\sum}_{k=0}^{p}g_{k}(\upbeta, \sigma^{2})\right)~~~~~~~~~~~~ \end{array} $$
(14)

where gk(β,σ2) is as specified before. □

Now, using our result from lemma B.1 in eq. 3, we get

$$ \begin{array}{@{}rcl@{}} &&P(\upbeta\mid y)\\ &&\overset{\upbeta}{\propto}\int P(y\mid \upbeta, \sigma^{2}) \frac{1}{\sqrt{(2\pi\sigma^{2}{\tau_{1}^{2}})^{p}}} \exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}{\tau_{1}^{2}}}\right) {\prod}_{j}\alpha_{j}\left( {\sum}_{k=0}^{p}g_{k}(\upbeta, \sigma^{2})\right)\\ &&P(\sigma^{2})d\sigma^{2} \end{array} $$
(15)
$$ \begin{array}{@{}rcl@{}} &&\overset{\upbeta}{\propto}{\sum}_{k=0}^{p}\int\frac{1}{\sigma^{2(n/2+p/2)}} \exp\left( -\frac{1}{\sigma^{2}} \left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}}\right)\right)\\ &&g_{k}(\upbeta, \sigma^{2})\frac{1}{\sigma^{2(a+1)}} \exp\left( -\frac{b}{\sigma^{2}}\right)d\sigma^{2} \end{array} $$
(16)
$$ \begin{array}{@{}rcl@{}} &&\overset{\upbeta}{\propto} {\sum}_{k=0}^{p}\int\frac{1}{\sigma^{2(n/2+p/2 + a +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)\\ &&g_{k}(\upbeta, \sigma^{2})d\sigma^{2}. \end{array} $$
(17)

Before evaluating the integrals, we need to show some identities. For that, we first need to define some expressions. Let, \(D_{\tau _{1}} = \tau _{1}^{-2}\mathbf {I}_{p}\), then we define

$$ \begin{array}{@{}rcl@{}} L_{\tau_{1}} &=& (\mathbf{x}^{T}\mathbf{x} + D_{\tau_{1}})^{-1}, \mu_{\tau_{1}} = L_{\tau_{1}}\mathbf{x}^{T}y,\\ r_{\tau_{1}} &=& \frac{y^{T}y - y^{T}\mathbf{x}L_{\tau_{1}}\mathbf{x}^{T}y}{2} +b, \text{ and } {\Sigma}^{-1}_{\tau_{1}} = \frac{n+2a}{2r_{\tau_{1}}}L_{\tau_{1}}^{-1}. \end{array} $$
(18)

We also use similar expressions using \(D_{\tau _{0}}\) and \(D_{j_{1},j_{2},\cdots ,j_{k}}\) where

$$ D_{\tau_{0}} = \tau_{0}^{-2}\mathbf{I}_{p} \text{ and } D_{j_{1},j_{2},\cdots,j_{k}} = \text{diag}((1-\mathbb{I}_{j_{1},j_{2},\cdots,j_{k}}(j))\tau_{1}^{-2} +\mathbb{I}_{j_{1},j_{2},\cdots,j_{k}}(j)\tau_{0}^{-2}). $$
(19)

Lemma B.2.

Let a/2 = n/2 + a. Then,

$$ \begin{array}{@{}rcl@{}} &&\int\frac{1}{\sigma^{2(n/2+p/2 + a +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)g_{k}(\upbeta, \sigma^{2})d\sigma^{2}\\ &&= {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2}h_{k}(\upbeta) \end{array} $$
(20)

where,

$$ \begin{array}{@{}rcl@{}} h_{0}(\upbeta) \!\!\!\!\!\!\!\!&&= \frac{\sqrt{|{\Sigma}_{\tau_{1}}|}} {r_{\tau_{1}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}(\mu_{\tau_{1}}, {\Sigma}_{\tau_{1}}) \end{array} $$
(21)
$$ \begin{array}{@{}rcl@{}} h_{p}(\upbeta) \!\!\!\!\!\!\!\!&&= \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\sqrt{|{\Sigma}_{\tau_{0}}|}} {r_{\tau_{0}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}(\mu_{\tau_{0}}, {\Sigma}_{\tau_{0}}) \end{array} $$
(22)

and for 1 ≤ kp − 1

$$ \begin{array}{@{}rcl@{}} h_{k}(\upbeta) &=& \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}} \frac{\sqrt{|{\Sigma}_{j_{1},j_{2},\cdots,j_{k}}|}} {r_{j_{1},j_{2},\cdots,j_{k}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}\\ &&\left( \mu_{j_{1},j_{2},\cdots,j_{k}}, {\Sigma}_{j_{1},j_{2},\cdots,j_{k}}\right). \end{array} $$
(23)

Proof 8.

We compute the integrals using the properties of inverse gamma distribution followed by some adjustments to obtain the expression of multivariate t distribution.

For k = 0, we have

$$ \begin{array}{@{}rcl@{}} &&{}\int\frac{1}{\sigma^{2(n/2+p/2 + a +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)g_{0}(\upbeta, \sigma^{2})d\sigma^{2}~~\\ &&{} = \int\frac{1}{\sigma^{2(a^{*}/2 + p/2 +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)d\sigma^{2} \end{array} $$
(24)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2+p/2)}{\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} +b\right)^{a^{*}/2 + p/2}} \end{array} $$
(25)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2+p/2)}{\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{{\upbeta}^{t} D_{\tau_{1}}\upbeta}{2} +b\right)^{a^{*}/2 + p/2}} \end{array} $$
(26)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2+p/2)}{\left( \frac{(\upbeta-\mu_{\tau_{1}})^{T}L_{\tau_{1}}^{-1} (\upbeta-\mu_{\tau_{1}})}{2} +r_{\tau_{1}}\right)^{a^{*}/2 + p/2}} \end{array} $$
(27)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2+p/2)}{r_{\tau_{1}}^{a^{*}/2 + p/2} \left( \frac{1}{a^{*}}\frac{(\upbeta-\mu_{\tau_{1}})^{T}L_{\tau_{1}}^{-1} (\upbeta-\mu_{\tau_{1}})}{2r_{\tau_{1}}/a^{*}} +1\right)^{a^{*}/2 + p/2}} \end{array} $$
(28)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2+p/2)}{r_{\tau_{1}}^{a^{*}/2 + p/2} \left( 1+\frac{1}{a^{*}}(\upbeta-\mu_{\tau_{1}})^{T} {\Sigma}^{-1}_{\tau_{1}}(\upbeta-\mu_{\tau_{1}})\right)^{a^{*}/2 + p/2}} \end{array} $$
(29)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2}\sqrt{|{\Sigma}_{\tau_{1}}|}} {r_{\tau_{1}}^{a^{*}/2 + p/2}} \\&&{}\frac{\Gamma(a^{*}/2+p/2)}{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2}\sqrt{|{\Sigma}_{\tau_{1}}|} \left( 1+\frac{1}{a^{*}}(\upbeta-\mu_{\tau_{1}})^{T} {\Sigma}^{-1}_{\tau_{1}}(\upbeta-\mu_{\tau_{1}})\right)^{a^{*}/2 + p/2}} \end{array} $$
(30)
$$ \begin{array}{@{}rcl@{}} &&{}= {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2}\frac{\sqrt{|{\Sigma}_{\tau_{1}}|}} {r_{\tau_{1}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}(\mu_{\tau_{1}}, {\Sigma}_{\tau_{1}}) \end{array} $$
(31)
$$ \begin{array}{@{}rcl@{}} &&{} = {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2} h_{0}(\upbeta). \end{array} $$
(32)

For k = p, we have

$$ \begin{array}{@{}rcl@{}} &&{}\int\frac{1}{\sigma^{2(n/2+p/2 + a +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)g_{p} (\upbeta,\sigma^{2})d\sigma^{2}\\ &&{} = \int\frac{1}{\sigma^{2(a^{*}/2 + p/2 +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right) \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}\\ &&{}\exp\left( -\frac{\|\upbeta\|^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right) {\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}}d\sigma^{2} \end{array} $$
(33)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \int\frac{1}{\sigma^{2(a^{*}/2 + p/2 +1)}}\\ &&\exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{0}^{2}}} +b \right)\right) \end{array} $$
(34)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\Gamma(a^{*}/2+p/2)}{\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{0}^{2}}}+b\right)^{a^{*}/2 + p/2}} \end{array} $$
(35)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\Gamma(a^{*}/2+p/2)}{\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{{\upbeta}^{T} D_{\tau_{0}}\upbeta}{2}+b\right)^{a^{*}/2 + p/2}} \end{array} $$
(36)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\Gamma(a^{*}/2+p/2)}{r_{\tau_{0}}^{a^{*}/2 + p/2} \left( 1+\frac{1}{a^{*}}(\upbeta-\mu_{\tau_{0}})^{T} {\Sigma}^{-1}_{\tau_{0}}(\upbeta-\mu_{\tau_{0}})\right)^{a^{*}/2 + p/2}} \end{array} $$
(37)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2}\sqrt{|{\Sigma}_{\tau_{0}}|}} {r_{\tau_{0}}^{a^{*}/2 + p/2}} \\&&{}\frac{\Gamma(a^{*}/2+p/2)}{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2}\sqrt{|{\Sigma}_{\tau_{0}}|} \left( 1+\frac{1}{a^{*}}(\upbeta-\mu_{\tau_{0}})^{T} {\Sigma}^{-1}_{\tau_{0}}(\upbeta-\mu_{\tau_{0}})\right)^{a^{*}/2 + p/2}} \end{array} $$
(38)
$$ \begin{array}{@{}rcl@{}} &&{}= {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2}\frac{{\tau_{1}^{p}}}{{\tau_{0}^{p}}}{\prod}_{j}\frac{1-\alpha_{j}}{\alpha_{j}} \frac{\sqrt{|{\Sigma}_{\tau_{0}}|}} {r_{\tau_{0}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}(\mu_{\tau_{0}}, {\Sigma}_{\tau_{0}}) \end{array} $$
(39)
$$ \begin{array}{@{}rcl@{}} &&{} = {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2} h_{p}(\upbeta). \end{array} $$
(40)

For 1 ≤ kp − 1, we have

$$ \begin{array}{@{}rcl@{}} &&{} \int\frac{1}{\sigma^{2(n/2+p/2 + a +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)g_{k}(\upbeta, \sigma^{2})d\sigma^{2}\\ &&{} = \int\frac{1}{\sigma^{2(a^{*}/2+p/2 +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}} + b\right)\right)\cdot\\ &&{}\frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}} \exp\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}+\cdots+{\upbeta}_{j_{k}}^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)d\sigma^{2} \end{array} $$
(41)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}}\cdot\\ &&{} \int\frac{1}{\sigma^{2(a^{*}/2 + p/2 +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{\|\upbeta\|^{2}}{2{\tau_{1}^{2}}}+b\right)\right) \\&&\exp\left( -\frac{{\upbeta}_{j_{1}}^{2}+{\upbeta}_{j_{2}}^{2}+\cdots+{\upbeta}_{j_{k}}^{2}}{2\sigma^{2}}\left( \frac{1}{{\tau_{0}^{2}}} -\frac{1}{{\tau_{1}^{2}}}\right)\right)d\sigma^{2} \end{array} $$
(42)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}}\cdot\\ &&{} \int\frac{1}{\sigma^{2(a^{*}/2 + p/2 +1)}} \exp\left( -\frac{1}{\sigma^{2}}\left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{{\upbeta}^{T} D_{j_{1},j_{2},\cdots,j_{k}}\upbeta}{2} + b\right)\right)d\sigma^{2} \end{array} $$
(43)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}} \frac{\Gamma(a^{*}/2 + p/2)}{ \left( \frac{\|y-\mathbf{x}\upbeta\|^{2}_{2}}{2} +\frac{{\upbeta}^{T} D_{j_{1},j_{2},\cdots,j_{k}}\upbeta}{2} + b\right)^{a^{*}/2+p/2}} \end{array} $$
(44)
$$ \begin{array}{@{}rcl@{}} &&{}= \frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}} \frac{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2}\sqrt{|{\Sigma}_{j_{1},j_{2},\cdots,j_{k}}|}} {r_{j_{1},j_{2},\cdots,j_{k}}^{a^{*}/2 + p/2}}\cdot\\ &&{} \frac{\Gamma(a^{*}/2+p/2)}{\Gamma(a^{*}/2)(a^{*}\pi)^{p/2} \sqrt{|{\Sigma}_{j_{1},j_{2},\cdots,j_{k}}|} \left( \!1 + \frac{1}{a^{*}}(\upbeta - \mu_{j_{1},j_{2},\cdots,j_{k}})^{T} {\Sigma}^{-1}_{j_{1},j_{2},\cdots,j_{k}}(\upbeta - \mu_{j_{1},j_{2},\cdots,j_{k}})\!\right)^{a^{*}/2 + p/2}} \end{array} $$
(45)
$$ \begin{array}{@{}rcl@{}} &&{}= {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2}\frac{{\tau_{1}^{k}}}{{\tau_{0}^{k}}} {\sum}_{j_{1}<\cdots<j_{k}}\frac{1-\alpha_{j_{1}}}{\alpha_{j_{1}}}\frac{1-\alpha_{j_{2}}}{\alpha_{j_{2}}} \cdots\frac{1-\alpha_{j_{k}}}{\alpha_{j_{k}}} \frac{\sqrt{|{\Sigma}_{j_{1},j_{2},\cdots,j_{k}}|}} {r_{j_{1},j_{2},\cdots,j_{k}}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}\\ &&(\mu_{j_{1},j_{2},\cdots,j_{k}}, {\Sigma}_{j_{1},j_{2},\cdots,j_{k}}) \end{array} $$
(46)
$$ \begin{array}{@{}rcl@{}} &&{}={\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2} h_{k}(\upbeta). \end{array} $$
(47)

Then, using identities from lemma B.2, we have

$$ \begin{array}{@{}rcl@{}} P(\upbeta \mid y) &&\overset{\upbeta}{\propto} {\Gamma}(a^{*}/2)(a^{*}\pi)^{p/2}{\sum}_{k=0}^{p} h_{k}(\upbeta) \end{array} $$
(48)
$$ \begin{array}{@{}rcl@{}} &&\overset{\upbeta}{\propto} {\sum}_{k=0}^{p} h_{k}(\upbeta). \end{array} $$
(49)

Now, for 1 ≤ kp − 1, we can rewrite h(k) so that

$$ h(k) = {\sum}_{\substack{\gamma \\ (\gamma_{j_{1}}=\cdots=\gamma_{j_{k}}=0)}} \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p-{\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}} {r_{\gamma}^{a^{*}/2 + p/2}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}), $$
(50)

where \(r_{\gamma } = \frac {y^{T}y - y^{T}\mathbf {x}L_{\gamma }\mathbf {x}^{T}y}{2} + b\) and \({\Sigma }_{\gamma } = \frac {a^{*}}{2r_{\gamma }}L^{-1}_{\gamma }\). Therefore,

$$ \begin{array}{@{}rcl@{}} &&{}P(\upbeta \mid y)\\ &&{}\overset{\upbeta}{\propto} {\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \right) \end{array} $$
(51)
$$ \begin{array}{@{}rcl@{}} && {}= \frac{{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \right)} {{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}}\right)}. \end{array} $$
(52)

This shows that joint posterior of β can be represented as a 2p component mixture of multivariate t-distribution, where each component corresponds to a particular combination of selected variables out of 2p possible combinations.

Now, by simplifying eq. 52, we have the joint posterior of β

$$ \begin{array}{@{}rcl@{}} &&{}P(\upbeta \mid y)\\ &&{} = \frac{{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \right)} {{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|{\Sigma}_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}}\right)} \end{array} $$
(53)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{2^{p/2}r_{\gamma}^{p/2} (a^{*})^{-p/2} \sqrt{|L_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \right)} {{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{2^{p/2}r_{\gamma}^{p/2} (a^{*})^{-p/2} \sqrt{|L_{\gamma}|}}{r^{a^{*}/2+p/2}_{\gamma}}\right)} \end{array} $$
(54)
$$ \begin{array}{@{}rcl@{}} &&{} = \frac{{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|L_{\gamma}|}}{r^{a^{*}/2}_{\gamma}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \right)} {{\sum}_{\gamma}\left( \left( \frac{\tau_{1}}{\tau_{0}}\right)^{p - {\sum}_{j}\gamma_{j}} {\prod}_{j}\left( \frac{1-\alpha_{j}}{\alpha_{j}}\right)^{1-\gamma_{j}} \frac{\sqrt{|L_{\gamma}|}}{r^{a^{*}/2}_{\gamma}}\right)} \end{array} $$
(55)
$$ \begin{array}{@{}rcl@{}} &&{}=\!\frac{{\sum}_{\gamma}\left( \! \left( {\prod}_{j} \alpha_{j}^{\gamma_{j}} (1 - \alpha_{j})^{1-\gamma_{j}}\right)\left( \frac{\sqrt{| L_{\gamma}|}} {\tau_{1}^{\sum\gamma_{j}} \tau_{0}^{(p-\sum\gamma_{j})}}\right) \frac{1}{\left( b + \frac{y^{T}y - {\mu_{\gamma}}^{T} L_{\gamma}^{-1} \mu_{\gamma}}{2}\right)^{n/2+a}} \mathcal{T}_{a^{*}}(\mu_{\gamma}, {\Sigma}_{\gamma}) \!\right)} {{\sum}_{\gamma}\left( \left( {\prod}_{j} \alpha_{j}^{\gamma_{j}} (1-\alpha_{j})^{1-\gamma_{j}}\right)\left( \frac{\sqrt{| L_{\gamma}|}} {\tau_{1}^{\sum\gamma_{j}} \tau_{0}^{(p-\sum\gamma_{j})}}\right) \frac{1}{\left( b + \frac{y^{T}y - {\mu_{\gamma}}^{T} L_{\gamma}^{-1} \mu_{\gamma}}{2}\right)^{n/2+a}}\right)}. \end{array} $$
(56)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Basu, T., Troffaes, M.C.M. & Einbeck, J. A Robust Bayesian Analysis of Variable Selection under Prior Ignorance. Sankhya A (2022). https://doi.org/10.1007/s13171-022-00287-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s13171-022-00287-2

Keywords

  • High dimensional data
  • Variable selection
  • Bayesian analysis
  • Imprecise probability.

PACS Nos

  • Primary 62J05; Secondary 62F15