Objective Bayesian analysis for exponential power regression models

Abstract

We develop objective Bayesian analysis for the linear regression model with random errors distributed according to the exponential power distribution. More specifically, we derive explicit expressions for three different Jeffreys priors for the model parameters. We show that only one of these Jeffreys priors leads to a proper posterior distribution. In addition, we develop fast posterior analysis based on Laplace approximations. Moreover, we show that our proposed Bayesian analysis compares favorably to a posterior analysis based on a competing noninformative prior. Finally, we illustrate our methodology with applications of the exponential power regression model to two different datasets.

A Appendix

Proof of Theorem 3.1

Using the results of Proposition 3.1 we have that

–:

For independence Jeffreys prior $\pi^{I_1}(\theta)$: marginal priors for β and (σ,p) are independent a priori such that $\pi^{I_1}(\beta,\sigma,p) = \pi^{I_1}(\beta) \pi^{I_1}(\sigma,p)$ and

$$ \begin{array}{rll} \pi^{I_1}(\sigma,p)&& \propto \sqrt{I_{\sigma \sigma}I_{pp}-I_{\sigma p}^2} \propto \frac{1}{\sigma p}\left\{ \left(1 + \frac{1}{p}\right)\Psi'\left(1 + 1/p\right) - 1 \right\}^{1/2}, \\ \pi^{I_1}(\beta) && \propto \sqrt{\det(I_{\beta \beta})} \propto 1. \end{array} $$

–:

For independence Jeffreys prior $\pi^{I_2}(\theta)$: let us consider independence Jeffreys prior such that $\pi^{I_2}(\theta) \propto \pi^{I_2}(\beta)\pi^{I_2}(\sigma)\pi^{I_2}(p)$, that is, taking each of these parameters as independent. Then, from the Fisher information matrix, $\pi^{I_2}(\beta) \propto 1$, $\pi^{I_2}(\sigma) \propto \sigma^{-1}$ and $\pi^{I_2}(p) \propto p^{-3/2}(1+p^{-1})^{1/2}\{\Psi'(1+p^{-1})\}^{1/2}$. Thus,

$$ \pi^{I_2}(\beta,\sigma,p) \propto \frac{1}{\sigma p^{3/2}}\left\{ \left(1 + \frac{1}{p}\right)\Psi'\left(1 + \frac{1}{p}\right)\right\}^{1/2}. $$

–:

For Jeffreys-rule prior: $\pi^J(\beta,\sigma,p) \propto \sqrt{\det(I(\theta))} = \sqrt{I_{\sigma \sigma}I_{pp}-I_{\sigma p}^2}\sqrt{\det(I_{\beta \beta})}$ where $\det(I_{\beta \beta}) \propto \left\{ {1}/{\sigma^2}\Gamma\left(1/p\right) \Gamma\left(2-1/p\right) \right\}^{k}$. Therefore,

$$ \pi^J(\beta,\sigma,p) \propto \frac{1}{\sigma^k}\left\{\Gamma\left(\frac{1}{p}\right)\Gamma\left(2-\frac{1}{p}\right) \right\}^{k/2}\pi^{I_1}(\sigma,p). $$

Proof of Lemma 3.2

Considering the integrated likelihood in (4.1), the integrated likelihood for p is

$$ \begin{array}{rll} L^I(p;y) &=& \int_{\mathbb{R}^k}L^I(\beta,p;y)\pi(\beta)d\beta\\ &\propto &\, p^{-1}\Gamma\left(\frac{n+a-1}{p}\right)\left\{\Gamma\left(1+\frac{1}{p}\right) \right\}^{-(n+a-1)}\\ &&\times\int_{\mathbb{R}^k}\left\{\sum\limits_{i=1}^{n}|y_i-x_i'\beta|^p\right\}^{-\frac{n+a-1}{p}}d\beta. \end{array} $$

Let us define the following two functions:

$$ \begin{array}{rll} h(\beta,p) &=& n\left(\max|y_i|+\mathop\sum\limits_{l=1}^k|\tilde{x}_l| |\beta_l| \right)^p \\ g(\beta,p) &=& \left\{ \begin{array}{ll} n\left|\bar{y}-\bar{x}'\beta^* \right|^p, & \mbox{if} \; \beta \in C_1, \\ n\left|\bar{y}-\bar{x}'\beta \right|^p, & \mbox{if} \; \beta \in C_2, \\ \end{array}\right. \end{array} $$

where $|\tilde{x}_l| = \max|x_{il}|$, C ₁ = {β ∈ ℝ, C ₂ = {β ∈ ℝand $\beta^* = {\displaystyle\arg\min_{\beta}} \sum_{i=1}^{n}|y_i-x_i'\beta|^p$. Note that $\sum_{i=1}^{n}|y_i-x_i'\beta^*|^p>0$ with probability 1 and, for example, if p = 2 then β ^* = (x′x)^− 1 x′y.

Note the following:

• (i)
  $$ \begin{array}{rll} \sum\limits_{i=1}^{n}|y_i-x_i'\beta|^p &\leq& \sum\limits_{i=1}^{n}(|y_i|+|x_i'\beta|)^p \leq \sum\limits_{i=1}^{n}\left(\max|y_i|+\sum\limits_{l=1}^k|x_{il}|\, |\beta_l|\right)^p \\ &\leq& \sum\limits_{i=1}^{n}\left(\max|y_i|+\sum\limits_{l=1}^k|\tilde{x}_l|\, |\beta_l| \right)^p = h(\beta,p). \end{array} $$

• (ii) For p ≥ 1 the function |·|^p is convex. Thus, by Jensen’s inequality

  $$ \sum\limits_{i=1}^n|y_i-x_i'\beta|^p \geq n\left|1/n\sum\limits_{i=1}^n (y_i- x_i'\beta)\right|^p = n|\bar{y}-\bar{x}'\beta|^p. $$

• (iii) Consider β ∈ C ₁. Using the definition of β ^* and result (ii) we have

  $$ \sum_{i=1}^n|y_i - x_i'\beta|^p \geq \sum_{i=1}^n|y_i-x_i'\beta^*|^p \geq n|\bar{y}-\bar{x}'\beta^*|^p = g(\beta,p). $$

• (iv) Consider β ∈ C ₂. Then by result (ii) we have

  $$ \sum_{i=1}^n|y_i - x_i'\beta|^p \geq n|\bar{y}-\bar{x}'\beta|^p = g(\mu,p). $$

• (v) Thus, by results (iii) and (iv),

  $$ \sum\limits_{i=^1}^n|y_i - x_i'\beta|^p \geq g(\beta,p). $$

Therefore, by results (i) and (v) above,

$$ \begin{array}{rll} \int_{\mathbb{R}^k}\{h(\beta,p)\}^{-\frac{n+a-1}{p}}d\beta &\leq& \int_{\mathbb{R}^k} \left\{\mathop\sum\limits_{i=1}^n|y_i-x_i'\beta|^p\right\}^{-\frac{n+a-1}{p}}d\beta\\ &\leq& \int_{\mathbb{R}^k}\{g(\beta,p)\}^{-\frac{n+a-1}{p}}d\beta. \end{array} $$

Let us now compute the left and right integral above. For the left integral and considering $\beta_{(-j)} = (\beta_{j+1}, \ldots, \beta_k)' \in \mathbb{R}^{k-j}$ for j = 1, ..., k − 1 we have:

$$ \begin{array}{rll} &&{\kern-6pt} \int_{\mathbb{R}^k}\{h(\beta,p)\}^{-\frac{n+a-1}{p}}d\beta \\ &&=\, 2\int\int_0^{\infty}\left\{ n^{\frac{1}{p}}\left(\max|y_i| + \sum_{l=2}^k|\tilde{x}_l||\beta_l| + |\tilde{x}_1|\beta_1\right)\right\}^{-(n+a-1)}d\beta_1 d\beta_{(-1)} \\ &&= \left.2\int\frac{n^{-\frac{n+a-1}{p}}\{max|y_i|+\sum\limits_{l=2}^k|\tilde{x}_l||\beta_l| + |\tilde{x}_1|\beta_1\}^{-(n+a-2)}}{|\tilde{x}_1|\{-(n+a-2)\}}\right|_{0}^{\infty}d\beta_{(-1)}\\ &&=\, 2 \int \frac{n^{-\frac{n+a-1}{p}} \{\max|y_i|+\sum_{l=2}^k|\tilde{x}_l||\beta_l|\}^{-(n+a-2)}}{|\tilde{x}_1|(n+a-2)}d\beta_{(-1)}. \end{array} $$

The integral above can be computed recursively for each element of the vector β _( − 1). The result is convergent as long as n > k + 1 − a, thus

$$ \int_{\mathbb{R}^k}\{h(\beta,p)\}^{-\frac{n+a-1}{p}}d\beta = n^{-\frac{n+a-1}{p}}m_1(y), $$

where $m_1(y) = \frac{2^k \Gamma(n+a-k-1)\{\max|y_i|\}^{-(n+a-1-k)}}{\Gamma(n+a-1) \prod_{l=1}^{k}|\tilde{x}_l|} > 0$, does not depend on p.

The right integral is

$$ \begin{array}{rll} \int_{\mathbb{R}^{k}}\{g(\beta,p)\}^{-\frac{n+a-1}{p}} d\beta &= & \int_{C_1}\left\{ n\left|\bar{y}-\bar{x}'\beta^* \right|^p \right\}^{-\frac{n+a-1}{p}}d\beta \\ &&+ \int_{C_2}\left\{ n\left|\bar{y}-\bar{x}'\beta \right|^p \right\}^{-\frac{n+a-1}{p}}d\beta\\ &= &\, n^{-\frac{n+a-1}{p}}m_2(y), \end{array} $$

where $m_2(y) = \int_{C_1} |\bar{y}-\bar{x}'\beta^*|^{-(n+a-1)}d\beta + \int_{C_2} |\bar{y}-\bar{x}'\beta|^{-(n+a-1)}d\beta$ does not depend on p.

Therefore, we have shown that for n > k + 1 − a

$$ m_1(y) \leq n^{\frac{n+a-1}{p}} \int_{-\infty}^{\infty}\left\{\sum\limits_{i=1}^{n}|y_i-x_i'\beta|^p\right\}^{-\frac{n+a-1}{p}}d\beta \leq m_2(y). $$

The result above will allow us to study the tail behavior of the integrated likelihood for p. First, note that from the series expansion of {Γ(z)}^− 1 (Abramowitz and Stegun, 1972, p.256) we obtain that {Γ(z)}^− 1 ≈ z from z close to 0. Thus, as p goes to ∞ we have Γ(1/p) ≈ p. In addition, considering the first-order Taylor expansion of logΓ(1 + z) around z = 0, logΓ(1 + z) ≈ Ψ(1)z, where Ψ(1) ≈ − 0.5772 is the digamma function evaluated at 1. Thus, $\Gamma(1+1/p) \approx \mbox{e}^{-0.5772/p}$ for large p. Therefore, for p → ∞,

$$ \begin{array}{rll} L^I(p;y) &\propto &\, p^{-1}\Gamma\left(\frac{n+a-1}{p}\right)\\ &&\times\left\{\Gamma\left(1+\frac{1}{p}\right)\right\}^{-(n+a-1)} \int_{\infty}^{\infty}\left\{\sum_{i=1}^{n}|y_i-x_i'\beta|^p\right\}^{-\frac{n+a-1}{p}}d\beta \\ &\approx &\, p^{-1}\frac{p}{n+a-1}\mbox{e}^{\Psi(1)(n+a-1)/p}O(n^{-(n+a-1)/p})\\ &= &\, O(\mbox{e}^{-\frac{n+a-1}{p}\{\log n - \Psi(1)\}})\\ &= &\, O(1). \end{array} $$