Saddlepoint expansions for GEL estimators

Abstract

A simple saddlepoint (SP) approximation for the distribution of generalized empirical likelihood (GEL) estimators is derived. Simulations compare the performance of the SP and other methods such as the Edgeworth and the bootstrap for special cases of GEL: continuous updating, empirical likelihood and exponential tilting estimators.

Appendix

This appendix lists the notations used in the paper and provides the regularity conditions underlying the results in the paper. It also provides proofs to those results not proven in the paper or which are not proven elsewhere.

Notational conventions:

For an $m\times n$ matrix $A$, $A^\top$ indicates its $n\times m$ transpose, $\Vert A\Vert =$ Trace$[AA^\top ]^{1/2}$. When the argument $\beta$ of a function is suppressed it is understood that the function is evaluated at $\beta _0$ so that, for example, $A\equiv A(\beta _0)$. The Kronecker product is defined in the usual way so that for $A=[a_{ij}]$ and a $p\times q$ matrix $B$, we have $A\otimes B=[a_{ij}B]$, an $mp\times nq$ matrix. The vectorization operator is defined in the usual way so that $\text {Vec}[A]$ denotes an $mn\times 1$ vector with the columns of $A$ stacked one upon each other.

The matrix of $\nu$’th order partial derivatives of a matrix $A(\beta )$ is indicated by $A^{(\nu )}(\beta )$. Specifically, if $A(\beta )$ is a $m\times 1$ vector function, $A^{(1)}(\beta )$ is the usual Jacobian whose $l$’th row contains the partials of the $l$’th element of $A(\beta )$. The matrices of higher derivatives are defined recursively so that the $j$’th element of the $l$’th row of $A^{(\nu )}(\beta )$ (a $m\times m^\nu$ matrix), is the $1\times m$ vector $a^{(\nu )}_{lj}(\beta )=\partial a^{(\nu -1)}_{lj}(\beta )/\partial \beta ^\top$. Two useful properties of these definitions, are that, if $a(\beta )=[a_j(\beta )]$ is a $m\times 1$ vector, then the $j$’th row of $a^{(1)}(\beta )$ contains the gradient of $a_j(\beta )$ and the $j$’th row of $a^{(2)}(\beta )$ contains the transposed vectorization of the Hessian matrix of $a_j(\beta )$.

A bar over a function indicates its expectation so that $\overline{A(\beta )}= E[A(\beta )]$. A tilde over a function indicates its deviation from its expectation so that $\widetilde{A(\beta )}= A(\beta )- \overline{A(\beta )}$. Also $Q = (E \left[ {q_i}^{(1)} \right] )^{-1}$ is evaluated at $\beta _0$ and is therefore treated as a constant.

Assumptions and proofs:

The result in Lemma 1 is proven in Rilstone et al. (1996) assuming $\widehat{\beta }$ is consistent, the data is i.i.d. and the following conditions satisfied for $s\ge 2$.

Assumption A

The $s$’th order derivatives of $q_i(\beta )$ exist in a neighborhood of $\beta _0$, $i=1,2,\ldots$ and $E\Vert q_i^{(s)}(\beta _0)\Vert ^{2}<\infty$.

Assumption B

For some neighborhood of $\beta _0$, $\left( \frac{1}{N}\sum q_i ^{(1)}(\beta )\right) ^{-1} = O_P(1)$.

Assumption C

$\left\| q_i^{(s)}(\beta ) - q_i^{(s)}(\beta _0)\right\| \le \Vert \beta -\beta _0\Vert M_i$ for some neighborhood of $\beta _0$ where $E| M_i| \le C<\infty$, $i=1,2,\ldots$.

Throughout this appendix we let $\bar{X}_m =\prod _{l=1}^m \frac{1}{N}\sum _{i=1}^N X_{li}$ denote the product of $m$ sample averages of mean-zero random variables.

Lemma A

Suppose $E \Vert X_{ji_j}\Vert ^m \le C<\infty$, $j=1,2,\ldots ,m$. Then

$$\begin{aligned} E[ \bar{X}_m] = {\left\{ \begin{array}{ll} O\left( N^{-m/2} \right) &{} {\text {for}} \quad m \quad {{even}} \\ O\left( N^{-(m+1)/2} \right) &{} {{for}} \quad m \quad {{odd}}. \end{array}\right. } \end{aligned}$$

Proof

Suppose $m>1$; otherwise $E\left[ \bar{X}_m \right] =0.$

$$\begin{aligned} E\left[ \bar{X}_m \right]&= N^{-m} \sum _{i_1=1}^N \sum _{i_2=1}^N \cdots \sum _{i_m=1}^N E\left[ X_{1i_1}X_{2i_2}\cdots X_{mi_m} \right] \nonumber \\&= {\left\{ \begin{array}{ll} N^{-m} O\left( N^{m /2} \right) = O\left( N^{-m /2} \right) &{} {\mathrm{for}} \quad m \quad {\mathrm{even}} \\ N^{-m} O\left( N^{ (m-1)/2}\right) =O\left( N^{ -(m+1)/2} \right) &{} {\mathrm{for}} \quad m \quad {\mathrm{odd}}. \end{array}\right. } \end{aligned}$$

(7.1)

The complete proof can be found in Kundhi and Rilstone (2012), Appendix B.

Proof to Lemma 3

The first moment of $\xi$ follows directly from the bias result in Rilstone et al. (1996). Since $E[\xi ]= O(N^{-1/2})$, we have $E\left[ ( \xi - E[\xi ])^2 \right] = E\left[ \xi ^2 \right] + O(N^{-1})$ and the second moment of $\xi$ is

$$\begin{aligned} E\left[ \xi ^2 \right]&= E\left[ \left( \xi _0 + \xi _{-{1/ 2}}\right) ^2 \right] \nonumber \\&= E\left[ \xi _0 ^2 \right] +T_N \end{aligned}$$

(7.2)

where

$$\begin{aligned} T_N&\equiv E\left[ \xi _{-{1/ 2}}^2 +2 \xi _{0} \xi _{-{1/ 2}} \right] . \end{aligned}$$

(7.3)

Applying Lemma A we see that $T_N$ is of the form and magnitude as follows

$$\begin{aligned} T_N&= N \left\{ E \left[ \bar{X}_4 + \bar{X}_3 \right] \right\} \nonumber \\&= N O\left( N^{-2}\right) \nonumber \\&= O\left( N^{-1}\right) \end{aligned}$$

(7.4)

so that

$$\begin{aligned} E\left[ \xi ^2 \right]&= E\left[ \xi _0 ^2 \right] + O\left( N^{-{1}}\right) \nonumber \\&= 1+ O\left( N^{-{1}}\right) . \end{aligned}$$

(7.5)

Finally, the third moment of $\xi$ is

$$\begin{aligned} E\left[ \xi ^3 \right] = E\left[ \xi _0 ^3 \right] +3 E\left[ \xi _0 ^2 \xi _{-{1/ 2}} \right] +T_N \end{aligned}$$

(7.6)

where

$$\begin{aligned} T_N \equiv E\left[ \xi _{-{1/ 2}}^3 + 3 \xi _{0} \xi _{-{1/ 2}} ^2 \right] . \end{aligned}$$

(7.7)

Applying Lemma A we see that $T_N$ is of the form and magnitude as follows

$$\begin{aligned} T_N&= N^{3/2} \left\{ E \left[ \bar{X}_6 + \bar{X}_5 \right] \right\} \nonumber \\&= N^{3/2} O\left( N^{-3}\right) \nonumber \\&= O\left( N^{-3/2}\right) \end{aligned}$$

(7.8)

so that

$$\begin{aligned} E\left[ \xi ^3 \right] = E\left[ \xi _0 ^3 \right] +3\,E\left[ \xi _0 ^2 \xi _{-{1/ 2}} \right] + O\left( N^{-{3/2}}\right) . \end{aligned}$$

(7.9)

Evaluating the first two terms we see first that

$$\begin{aligned} E\left[ \xi _0 ^3 \right]&= -{N^{3/2}\over \eta ^{3/2}} E \left[ \frac{1}{N}\sum _{i=1}^N \tau ^\top d_i\right] ^3\nonumber \\&= -\frac{1}{\sqrt{N}}\frac{E\left[ (\tau ^\top d_i)^3\right] }{\eta ^{3/2}}. \end{aligned}$$

(7.10)

From the random sampling assumption and Lemma A

$$\begin{aligned} E\left[ \xi _{0} ^2 \xi _{-1/2} \right]&= \frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_1 \left( { {\tau ^\top d^{(1)}_2}} d_2-\frac{1}{2} {\tau ^\top \overline{ d^{(2)}_1}}\left( d_2\otimes d_2 \right) \right) \right] \nonumber \\&\quad +\frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_2 \left( { {\tau ^\top d^{(1)}_1}} d_2-\frac{1}{2} {\tau ^\top \overline{d ^{(2)}_1}}\left( d_1\otimes d_2 \right) \right) \right] \nonumber \\&\quad + \frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_2 \left( { {\tau ^\top d^{(1)}_2}} d_1-\frac{1}{2} \tau ^\top {\overline{d ^{(2)}_1}}\left( d_2\otimes d_1\right) \right) \right] + O\left( N^{-3/2}\right) \nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}}\left\{ \eta \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2}\eta \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right\} \nonumber \\&\quad + \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ 2 \tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - \tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) \right\} + O\left( N^{-3/2}\right) \nonumber \\ \end{aligned}$$

(7.11)

We can therefore write the third cumulant as

$$\begin{aligned} E\left[ \left( \xi - E\left[ \xi \right] \right) ^3 \right]&= E\left[ \xi ^3 \right] -3 E\left[ \xi ^2 \right] E\left[ \xi \right] +2 \left( E\left[ \xi \right] \right) ^3\nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ - E\left[ (\tau ^\top d_i)^3\right] + 3\left( \eta \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2}\eta \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right) \right\} \nonumber \\&+ \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} 3 \left\{ 2\tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - \tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) \right\} \nonumber \\&-3 {1\over \sqrt{N}}{{1\over \sqrt{\eta }} }\left\{ \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2} \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right\} + O\left( N^{-3/2}\right) \nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ 6 \tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - 3\tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) - E[(\tau ^\top d_1)^3] \right\} \nonumber \\&+\,\, O\left( N^{-3/2}\right) . \end{aligned}$$

(7.12)

Proof to Proposition 1

We note that

$$\begin{aligned} K^{[1]}( t(x) ) -\widehat{K}^{[1]}(\widehat{t})= x-x= 0 \end{aligned}$$

(7.13)

so that

$$\begin{aligned}&K^{[1]}( \widehat{t}) -\widehat{K}^{[1]}(\widehat{t}) = (t-\widehat{t}) K^{[2]}( \widehat{t})\end{aligned}$$

(7.14)

$$\begin{aligned}&(t- \widehat{t}) = \frac{K^{[1]}( \widehat{t}) -\widehat{K}^{[1]}(\widehat{t})}{ K^{[2]}( \widehat{t})} = o(N^{-1/2}) \end{aligned}$$

(7.15)

Similarly,

$$\begin{aligned} K^{[j]} ( t(x)) -\widehat{K} ^{[j]}(\widehat{t})&= K^{[j]} ( \widehat{t}) -\widehat{K}^{[j]} (\widehat{t}) + (t(x) -\widehat{t}) K^{[j+1]}( \bar{t})\nonumber \\&= o(N^{-1/2}),\qquad j=0,1,2 \end{aligned}$$

(7.16)

where $\bar{t}$ is a mean value.

$$\begin{aligned} f(x;K) -\widehat{f}(x)&= \frac{1}{\sqrt{2\pi }} \left( \frac{1}{\sqrt{ K^{[2]}( {t})}}\exp \left\{ K( t(x) ) - {t}(x)x\right\} \right. \nonumber \\&\left. - \frac{1}{\sqrt{ \widehat{K}^{[2]}(\widehat{t})}}\exp \left\{ K(\widehat{t}) - \widehat{t}x\right\} \right) \nonumber \\&= o\left( N^{-1/2}\right) . \end{aligned}$$

(7.17)