Bayesian empirical likelihood inference and order shrinkage for autoregressive models

Abstract

This paper considers the Bayesian empirical likelihood (BEL) inference and order shrinkage for a class of sparse autoregressive models without assuming the distributions for the errors. By introducing a nonparametric likelihood, parameters’ point and interval estimators, as well as some asymptotic properties of the estimators are obtained. By introducing a spike-and-slab prior, the order and the non-zero autoregressive coefficients of the model can be easily and accurately determined together via the Markov Chain Monte Carlo (MCMC) techniques. Simulation studies are conducted to evaluate the proposed methods. Finally, a real data example of the US industrial production index data set is applied to show the good performances of the BEL methods.

Appendix

Proof of Theorem 3.1

It it easy to see that (S1) and (S2) are equivalent. Therefore, we only prove (S2). Recall that ${\varvec{Y}}_{t}=\left( y_{t}, \dots , y_{t-p+1}\right) ^{\textsf {T}}$ and ${\varvec{Z}}_n=\sum _{t=p+1}^n {\varvec{Y}}_{t-1}{\varvec{Y}}_{t-1}^{\textsf {T}}$. Then, we have ${\varvec{\Sigma }}_n=\sum _{t=p+1}^n {\varvec{m}}_t({\varvec{\beta }}){\varvec{m}}_t({\varvec{\beta }})^{\textsf {T}} =\sum _{t=p+1}^n (y_t-{\varvec{\beta }}^{\textsf {T}}{\varvec{Y}}_{t-1})^2 {\varvec{Y}}_{t-1}{\varvec{Y}}_{t-1}^{\textsf {T}}$. By Theorem 1.1 in Billingsley (1961) and continuous mapping theorem, we have

$$\begin{aligned} \frac{1}{n}\sum _{t=p+1}^n {\varvec{Y}}_{t-1}{\varvec{Y}}_{t-1}^{\textsf {T}} \overset{P}{\longrightarrow }E({\varvec{Y}}_{p}{\varvec{Y}}_{p}^{\textsf {T}}):={\varvec{C}}_1 <\infty ,~n\rightarrow \infty , \end{aligned}$$

and

$$\begin{aligned}&\frac{1}{n}\sum _{t=p+1}^n (y_t-{\varvec{\beta }}^{\textsf {T}}{\varvec{Y}}_{t-1})^2 {\varvec{Y}}_{t-1}{\varvec{Y}}_{t-1}^{\textsf {T}} \overset{P}{\longrightarrow }E((y_{p+1}-{\varvec{\beta }}^{\textsf {T}} {\varvec{Y}}_{p})^2{\varvec{Y}}_{p}{\varvec{Y}}_{p}^{\textsf {T}})\\&\qquad :={\varvec{C}}_2 <\infty ,~n\rightarrow \infty , \end{aligned}$$

where ${\varvec{C}}_1$ and ${\varvec{C}}_2$ are constant matrices. Note that as $n\rightarrow \infty$, $\frac{1}{n}{\varvec{\Lambda }}^{-1}\rightarrow {\varvec{0}}.$ Thus,

$$\begin{aligned} \frac{1}{n}({\varvec{Z}}_n^{\textsf {T}}{\varvec{\Sigma }}^{-1}{\varvec{Z}}_n+{\varvec{\Lambda }}^{-1}) \overset{P}{\longrightarrow }{\varvec{C}}_1^{\textsf {T}}{\varvec{C}}_2^{-1}{\varvec{C}}_1,~n\rightarrow \infty . \end{aligned}$$

Thus, by continuous mapping theorem, we have as $n\rightarrow \infty$,

$$\begin{aligned} ({\varvec{Z}}_n^{\textsf {T}}{{\varvec{\Sigma }}}^{-1}{\varvec{Z}}_n+{\varvec{\Lambda }}^{-1})^{-1}{\varvec{\Lambda }}^{-1} \overset{P}{\longrightarrow }\frac{1}{n}({\varvec{C}}_1^{\textsf {T}}{\varvec{C}}_2^{-1}{\varvec{C}}_1)^{-1}{\varvec{\Lambda }}^{-1} \overset{P}{\longrightarrow }{\varvec{0}}. \end{aligned}$$

Since $\hat{{\varvec{\Sigma }}}$ is a consistent estimator of ${\varvec{\Sigma }}$, by Slutsky’s theorem, (S2) holds.

The proof is complete. $\square$

The full conditional density of $\beta _{i} | {\varvec{Y}}, {\varvec{\theta }}, \eta , {\varvec{\beta }}_{-i}$ is:

$$\begin{aligned} \begin{aligned}&f\left( \beta _{i} | {\varvec{Y}}, {\varvec{\theta }}, \eta , {\varvec{\beta }}_{-i}\right) \\&\quad \propto \exp \left\{ -\frac{1}{2} \left( \beta _{i}-\mu _{i}\right) ^{2}V_{i} \right\} \pi \left( \beta _{i} | \theta _{i}, \eta \right) \\&\quad \propto \exp \left\{ -\frac{1}{2} \left( \beta _{i}-\mu _{i}\right) ^{2}V_{i} \right\} \left[ \theta _{i} I_{\left\{ \beta _{i}=0\right\} }+\left( 1-\theta _{i}\right) I_{\{\beta _i \ne 0\}} \frac{1}{\sqrt{2 \pi \eta ^{-1}}}\exp \left( -\frac{\beta _{i}^{2}}{2} \eta \right) \right] \\&\quad \propto \theta _{i} \exp \left\{ -\frac{1}{2} \mu _{i}^{2}V_{i} \right\} I_{\left\{ \beta _{i}=0\right\} } +\left( 1-\theta _{i}\right) \exp \left\{ -\frac{1}{2} \left( \beta _{i}-\mu _{i}\right) ^{2}V_{i} \right\} \frac{1}{\sqrt{2 \pi \eta ^{-1}}}\\&\qquad \exp \left( -\frac{\beta _{i}^{2}}{2} \eta \right) I_{\{\beta _i \ne 0\}} \\&\quad \propto \theta _{i} I_{\left\{ \beta _{i}=0\right\} }+\left( 1-\theta _{i}\right) \exp \left\{ \frac{1}{2} \frac{V_i^2\mu _i^2}{\eta +V_i}\right\} \sqrt{ \frac{\eta }{V_i+\eta } } \sqrt{\frac{V_i+\eta }{ 2 \pi }} \exp \\&\qquad \left\{ -\frac{1}{2}(\eta +V_i) \left( \beta _i- \frac{V_i\mu _i}{\eta +V_i} \right) ^2 \right\} I_{\{\beta _i \ne 0\}} \\&\quad \propto \tilde{\theta }_{i} I_{\left\{ \beta _{i}=0\right\} }+\left( 1-\tilde{\theta }_{i}\right) N\left( \frac{V_{i} \mu _{i}}{V_{i}+\eta }, \frac{1}{V_{i}+\eta }\right) I_{\left\{ \beta _{i}\ne 0\right\} }, \end{aligned} \end{aligned}$$

where $\tilde{\theta }_{i}= \theta _{i}\left( \theta _{i}+\left( 1-\theta _{i}\right) \exp \left\{ \frac{1}{2} \frac{V_i^2\mu _i^2}{\eta +V_i}\right\} \sqrt{ \frac{\eta }{V_i+\eta } } \right) ^{-1}$.