High-dimensional properties for empirical priors in linear regression with unknown error variance

Abstract

We study full Bayesian procedures for high-dimensional linear regression. We adopt data-dependent empirical priors introduced in Martin et al. (Bernoulli 23(3):1822–1847, 2017). In their paper, these priors have nice posterior contraction properties and are easy to compute. Our paper extend their theoretical results to the case of unknown error variance . Under proper sparsity assumption, we achieve model selection consistency, posterior contraction rates as well as Bernstein von-Mises theorem by analyzing multivariate t-distribution.

Appendix A: Tail bounds for the Chi squared distribution

Lemma 2

For any $a>0$, we have

$$\begin{aligned} P\left( \vert \chi ^2_p -p\vert >a\right) \le 2 \exp \left( -\frac{a^2}{4p}\right) . \end{aligned}$$

Proof: See Lemma 4.1 of Cao et al. (2020).

Lemma 3

(i) For any $c>0$, we have

$$\begin{aligned} P\left( \chi ^2_p(\lambda )-(p+\lambda )>c\right) \le \exp \left( -\frac{p}{2} \left\{ \frac{c}{p+\lambda }-log\left( 1+\frac{c}{p+\lambda }\right) \right\} \right) . \end{aligned}$$

(ii) for $\omega <1$ then

$$\begin{aligned} P(\chi ^2_p(\lambda ) \le \omega \lambda ) \le c_1 \lambda ^{-1} \exp \{-\lambda (1-\omega )^2/8\}, \end{aligned}$$

where $c_1>0$ is a constant.

(iii) For any $c>0$, $P(\chi ^2_p(\lambda )-p \le -c) \le \exp \left( -\frac{c^2}{4p}\right)$.

Proof: See (Cao et al. 2020) Lemma 4.2 for (i) and for (ii) on (Shin et al. 2019). (iii) follows from the fact if $U \sim \chi ^2_p(\lambda )$, then $P(U>c)$ is strictly increasing in $\lambda$ for fixed p and $c>0$. Hence

$$\begin{aligned} P\left( \chi ^2_p(\lambda )-p \le -c\right) \le P\left( \chi ^2_p-p \le -c\right) \le \exp \left( -\frac{c^2}{4p}\right) . \end{aligned}$$