Skip to main content

Saddlepoint expansions for GEL estimators

Abstract

A simple saddlepoint (SP) approximation for the distribution of generalized empirical likelihood (GEL) estimators is derived. Simulations compare the performance of the SP and other methods such as the Edgeworth and the bootstrap for special cases of GEL: continuous updating, empirical likelihood and exponential tilting estimators.

This is a preview of subscription content, access via your institution.

Notes

  1. 1.

    See also Rilstone and Ullah (2005) for a correction to the second-order variance term.

  2. 2.

    We note that Newey and Smith (2004) and Kundhi and Rilstone (2012) examined the bias and variance and Edgeworth, respectively, of GEL estimators specifically although results were available for similar classes of estimators.

  3. 3.

    We use the abbreviation SP or capitalize “Saddlepoint” when we refer to the SP approximation as distinguished to the stationary saddle point of \(\mathcal{L}(\theta ,\lambda )\).

  4. 4.

    Henceforth all conditions are for \(i=1,\ldots ,N\) and summations are over the same range.

  5. 5.

    Also see Huzurbazar and Williams (2010) in this regard.

  6. 6.

    Under certain conditions renormalizing the SP approximation further reduces the size of the relative error as argued by Daniels (1980).

  7. 7.

    See also Mittelhammer et al. (2000).

  8. 8.

    The number of bootstrap re-samples is chosen such that \(\alpha (B + 1)\) is an integer as in Davidson and MacKinnon (2000).

References

  1. Barndorff-Nielson OE, Cox DR (1989) Asymptotic techniques for use in statistics. Chapman and Hall, New York

    Book  Google Scholar 

  2. Bhattacharya RN, Ghosh JK (1978) On the validity of the formal Edgeworth expansion. Ann Stat 6(2):434–451

    Article  MATH  MathSciNet  Google Scholar 

  3. Butler RW (2007) Saddlepoint approximations with applications. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  4. Czellar V, Ronchetti E (2010) Accurate and robust tests for indirect inference. Biometrika 97(3):621–630

    Article  MATH  MathSciNet  Google Scholar 

  5. Daniels HE (1954) Saddlepoint approximations in statistics. Ann Math Stat 25:631–650

    Article  MATH  MathSciNet  Google Scholar 

  6. Daniels HE (1980) Exact saddlepoint approximations. Biometrika 67:59–63

    Article  MATH  MathSciNet  Google Scholar 

  7. Daniels HE (1983) Saddlepoint approximations for estimating equations. Biometrika 70:89–96

    Article  MATH  MathSciNet  Google Scholar 

  8. Davison AC, Hinkley DV (1988) Saddlepoint approximations in resampling methods. Biometrika 75(3):417–431

    Article  MATH  MathSciNet  Google Scholar 

  9. Davidson R, MacKinnon JG (2000) Bootstrap tests: how many bootstraps? Econ Rev 19:55–68

    Article  MATH  MathSciNet  Google Scholar 

  10. Easton GS, Ronchetti E (1986) General saddlepoint approximations with applications to L statistics. J Am Stat Assoc 81:420–430

    MATH  MathSciNet  Google Scholar 

  11. Field CA (1982) Small sample asymptotic expansions for multivariate M- estimates. Ann Stat 10:672–689

    Article  MATH  MathSciNet  Google Scholar 

  12. Field CA, Hampel FR (1982) Small-sample asymptotic distributions of M-estimators of location. Biometrika 69:29–46

    Article  MATH  MathSciNet  Google Scholar 

  13. Field CA, Ronchetti E (1990) Small sample asymptotics. Institute of Mathematical Statistics. Lecture notes monograph series. Institute of Mathematical Statistics, Hayward [147]

  14. Feuerverger A (1989) On the empirical saddlepoint approximation. Biometrika 76(3):457–464

    Article  MATH  MathSciNet  Google Scholar 

  15. Ghosh JK, Sinha BK, Wieand HS (1980) Second-order efficiency of the MLE with respect to any bowl-shaped loss function. Ann Math Stat 8:506–521

    Article  MATH  MathSciNet  Google Scholar 

  16. Hansen LP, Heaton J, Yaron A (1996) Finite sample properties of some alternative GMM estimators. J Bus Econ Stat 14:262–280

    Google Scholar 

  17. Huzurbazar AV, Williams BJ (2010) Incorporating cavariates in flowgraph models: applications to recurrent event data. Technometrics 52:198–208

    Article  MathSciNet  Google Scholar 

  18. Jensen JL (1995) Saddlepoint approximations. Oxford University Press, USA

    MATH  Google Scholar 

  19. Jing BY, Feuerverger A, Robinson J (1994) On the bootstrap saddlepoint approximations. Biometrika 81(1):211–215

    Article  MATH  MathSciNet  Google Scholar 

  20. Kundhi G, Rilstone P (2008) The third-order bias of nonlinear estimators. Commun Stat Theor Methods 37(16):2617–2633

    Article  MATH  MathSciNet  Google Scholar 

  21. Kundhi G, Rilstone P (2012) Edgeworth expansions for GEL estimators. J Multivar Anal 106:118–146

    Article  MATH  MathSciNet  Google Scholar 

  22. Kundhi G, Rilstone P (2013) Edgeworth and saddlepoint expansions for nonlinear estimators. Econ Theory 29(5):1057–1078

  23. Lô SN, Ronchetti E (2012) Robust small sample accurate inference in moment condition models. Comput Stat Data Anal 56:3182–3197

    Article  MATH  Google Scholar 

  24. Lugannani R, Rice SO (1980) Saddlepoint approximations for the distributions of the sum of random variables. Adv Appl Probab 12:475–490

    Article  MATH  MathSciNet  Google Scholar 

  25. Ma Y, Ronchetti E (2011) Saddlepoint test in measurement error models. J Am Stat Assoc 106(493):147–156

    Article  MathSciNet  Google Scholar 

  26. McCullagh P (1987) Tensor methods in statistics. Chapman and Hall, New York

    MATH  Google Scholar 

  27. Mittelhammer RC, Judge GG, Miller DJ (2000) Econometric foundations. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  28. Monti AC, Ronchetti E (1993) On the relationship between empirical likelihood and empirical approximations for multivariate M-estimators. Biometrika 80:329–338

    Article  MATH  MathSciNet  Google Scholar 

  29. Newey WK, Smith R (2004) Higher order properties of GMM and generalized empirical likelihood estimators. Econometrica 72(1):219–255

    Article  MATH  MathSciNet  Google Scholar 

  30. Phillips PCB (1977) A general theorem in the theory of asymptotic expansions as approximations to the finite sample distributions of econometric estimators. Econometrica 45(6):1517–1534

    Article  MATH  MathSciNet  Google Scholar 

  31. Pfanzagl J, Wefelmeyer W (1978) A third-order optimum property of the maximium likelihood estimator. J Multivar Anal 10:1–29

    Article  MathSciNet  Google Scholar 

  32. Qin J, Lawless J (1994) Empirical Likelihood and general estimating equations. Ann Stat 22(1):300–325

    Article  MATH  MathSciNet  Google Scholar 

  33. Rilstone P, Srivastava VK, Ullah A (1996) The second-order bias and mean squared error of nonlinear estimators. J Econom 75(2):369–395

    Article  MATH  MathSciNet  Google Scholar 

  34. Rilstone P, Ullah A (2005) Corrigendum to: the second-order bias and mean squared error of non-linear estimators. J Econom 124(1):203–204

    Article  MathSciNet  Google Scholar 

  35. Reid N (1988) Saddlepoint methods and statistical inference. Stat Sci 3(2):213–238

    Article  MATH  Google Scholar 

  36. Robinson J, Ronchetti E, Young GA (2003) Saddlepoint approximations and tests based on multivariate M-estimates. Ann Stat 31:1154–1169

    Article  MATH  MathSciNet  Google Scholar 

  37. Ronchetti E, Welsh AH (1994) Empirical saddlepoint approximations for multivariate M-estimators. J R Stat Soc Ser B 56:313–326

    MATH  MathSciNet  Google Scholar 

  38. Rothenberg TJ (1984) Approximating the distributions of econometric estimators and test statistics. In: Griliches Z, Intriligator MD (eds) Handbook of econometrics, vol 2, chap 15. Elsevier Science Inc., New York

    Google Scholar 

  39. Schennach SM (2007) Point estimation with exponentially tilted empirical likelihood. Ann Stat 35(2):634–672

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Gubhinder Kundhi.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 33 KB)

Supplementary material 2 (pdf 515 KB)

Appendix

Appendix

This appendix lists the notations used in the paper and provides the regularity conditions underlying the results in the paper. It also provides proofs to those results not proven in the paper or which are not proven elsewhere.

Notational conventions:

For an \(m\times n\) matrix \(A\), \(A^\top \) indicates its \(n\times m\) transpose, \(\Vert A\Vert =\) Trace\([AA^\top ]^{1/2}\). When the argument \(\beta \) of a function is suppressed it is understood that the function is evaluated at \(\beta _0\) so that, for example, \(A\equiv A(\beta _0)\). The Kronecker product is defined in the usual way so that for \(A=[a_{ij}]\) and a \(p\times q\) matrix \(B\), we have \(A\otimes B=[a_{ij}B]\), an \(mp\times nq\) matrix. The vectorization operator is defined in the usual way so that \(\text {Vec}[A]\) denotes an \(mn\times 1\) vector with the columns of \(A\) stacked one upon each other.

The matrix of \(\nu \)’th order partial derivatives of a matrix \(A(\beta )\) is indicated by \( A^{(\nu )}(\beta )\). Specifically, if \(A(\beta )\) is a \(m\times 1\) vector function, \( A^{(1)}(\beta )\) is the usual Jacobian whose \(l\)’th row contains the partials of the \(l\)’th element of \(A(\beta )\). The matrices of higher derivatives are defined recursively so that the \(j\)’th element of the \(l\)’th row of \( A^{(\nu )}(\beta )\) (a \(m\times m^\nu \) matrix), is the \(1\times m\) vector \(a^{(\nu )}_{lj}(\beta )=\partial a^{(\nu -1)}_{lj}(\beta )/\partial \beta ^\top \). Two useful properties of these definitions, are that, if \(a(\beta )=[a_j(\beta )]\) is a \(m\times 1\) vector, then the \(j\)’th row of \( a^{(1)}(\beta )\) contains the gradient of \(a_j(\beta )\) and the \(j\)’th row of \( a^{(2)}(\beta )\) contains the transposed vectorization of the Hessian matrix of \(a_j(\beta )\).

A bar over a function indicates its expectation so that \(\overline{A(\beta )}= E[A(\beta )]\). A tilde over a function indicates its deviation from its expectation so that \(\widetilde{A(\beta )}= A(\beta )- \overline{A(\beta )}\). Also \( Q = (E \left[ {q_i}^{(1)} \right] )^{-1}\) is evaluated at \(\beta _0\) and is therefore treated as a constant.

Assumptions and proofs:

The result in Lemma 1 is proven in Rilstone et al. (1996) assuming \(\widehat{\beta }\) is consistent, the data is i.i.d. and the following conditions satisfied for \(s\ge 2\).

Assumption A

The \(s\)’th order derivatives of \(q_i(\beta )\) exist in a neighborhood of \(\beta _0\), \(i=1,2,\ldots \) and \(E\Vert q_i^{(s)}(\beta _0)\Vert ^{2}<\infty \).

Assumption B

For some neighborhood of \(\beta _0\), \(\left( \frac{1}{N}\sum q_i ^{(1)}(\beta )\right) ^{-1} = O_P(1)\).

Assumption C

\( \left\| q_i^{(s)}(\beta ) - q_i^{(s)}(\beta _0)\right\| \le \Vert \beta -\beta _0\Vert M_i \) for some neighborhood of \(\beta _0\) where \(E| M_i| \le C<\infty \), \(i=1,2,\ldots \).

Throughout this appendix we let \(\bar{X}_m =\prod _{l=1}^m \frac{1}{N}\sum _{i=1}^N X_{li}\) denote the product of \(m\) sample averages of mean-zero random variables.

Lemma A

Suppose \(E \Vert X_{ji_j}\Vert ^m \le C<\infty \), \(j=1,2,\ldots ,m\). Then

$$\begin{aligned} E[ \bar{X}_m] = {\left\{ \begin{array}{ll} O\left( N^{-m/2} \right) &{} {\text {for}} \quad m \quad {{even}} \\ O\left( N^{-(m+1)/2} \right) &{} {{for}} \quad m \quad {{odd}}. \end{array}\right. } \end{aligned}$$

Proof

Suppose \(m>1\); otherwise \(E\left[ \bar{X}_m \right] =0.\)

$$\begin{aligned} E\left[ \bar{X}_m \right]&= N^{-m} \sum _{i_1=1}^N \sum _{i_2=1}^N \cdots \sum _{i_m=1}^N E\left[ X_{1i_1}X_{2i_2}\cdots X_{mi_m} \right] \nonumber \\&= {\left\{ \begin{array}{ll} N^{-m} O\left( N^{m /2} \right) = O\left( N^{-m /2} \right) &{} {\mathrm{for}} \quad m \quad {\mathrm{even}} \\ N^{-m} O\left( N^{ (m-1)/2}\right) =O\left( N^{ -(m+1)/2} \right) &{} {\mathrm{for}} \quad m \quad {\mathrm{odd}}. \end{array}\right. } \end{aligned}$$
(7.1)

The complete proof can be found in Kundhi and Rilstone (2012), Appendix B.

Proof to Lemma 3

The first moment of \(\xi \) follows directly from the bias result in Rilstone et al. (1996). Since \(E[\xi ]= O(N^{-1/2})\), we have \( E\left[ ( \xi - E[\xi ])^2 \right] = E\left[ \xi ^2 \right] + O(N^{-1}) \) and the second moment of \(\xi \) is

$$\begin{aligned} E\left[ \xi ^2 \right]&= E\left[ \left( \xi _0 + \xi _{-{1/ 2}}\right) ^2 \right] \nonumber \\&= E\left[ \xi _0 ^2 \right] +T_N \end{aligned}$$
(7.2)

where

$$\begin{aligned} T_N&\equiv E\left[ \xi _{-{1/ 2}}^2 +2 \xi _{0} \xi _{-{1/ 2}} \right] . \end{aligned}$$
(7.3)

Applying Lemma A we see that \(T_N\) is of the form and magnitude as follows

$$\begin{aligned} T_N&= N \left\{ E \left[ \bar{X}_4 + \bar{X}_3 \right] \right\} \nonumber \\&= N O\left( N^{-2}\right) \nonumber \\&= O\left( N^{-1}\right) \end{aligned}$$
(7.4)

so that

$$\begin{aligned} E\left[ \xi ^2 \right]&= E\left[ \xi _0 ^2 \right] + O\left( N^{-{1}}\right) \nonumber \\&= 1+ O\left( N^{-{1}}\right) . \end{aligned}$$
(7.5)

Finally, the third moment of \(\xi \) is

$$\begin{aligned} E\left[ \xi ^3 \right] = E\left[ \xi _0 ^3 \right] +3 E\left[ \xi _0 ^2 \xi _{-{1/ 2}} \right] +T_N \end{aligned}$$
(7.6)

where

$$\begin{aligned} T_N \equiv E\left[ \xi _{-{1/ 2}}^3 + 3 \xi _{0} \xi _{-{1/ 2}} ^2 \right] . \end{aligned}$$
(7.7)

Applying Lemma A we see that \(T_N\) is of the form and magnitude as follows

$$\begin{aligned} T_N&= N^{3/2} \left\{ E \left[ \bar{X}_6 + \bar{X}_5 \right] \right\} \nonumber \\&= N^{3/2} O\left( N^{-3}\right) \nonumber \\&= O\left( N^{-3/2}\right) \end{aligned}$$
(7.8)

so that

$$\begin{aligned} E\left[ \xi ^3 \right] = E\left[ \xi _0 ^3 \right] +3\,E\left[ \xi _0 ^2 \xi _{-{1/ 2}} \right] + O\left( N^{-{3/2}}\right) . \end{aligned}$$
(7.9)

Evaluating the first two terms we see first that

$$\begin{aligned} E\left[ \xi _0 ^3 \right]&= -{N^{3/2}\over \eta ^{3/2}} E \left[ \frac{1}{N}\sum _{i=1}^N \tau ^\top d_i\right] ^3\nonumber \\&= -\frac{1}{\sqrt{N}}\frac{E\left[ (\tau ^\top d_i)^3\right] }{\eta ^{3/2}}. \end{aligned}$$
(7.10)

From the random sampling assumption and Lemma A

$$\begin{aligned} E\left[ \xi _{0} ^2 \xi _{-1/2} \right]&= \frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_1 \left( { {\tau ^\top d^{(1)}_2}} d_2-\frac{1}{2} {\tau ^\top \overline{ d^{(2)}_1}}\left( d_2\otimes d_2 \right) \right) \right] \nonumber \\&\quad +\frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_2 \left( { {\tau ^\top d^{(1)}_1}} d_2-\frac{1}{2} {\tau ^\top \overline{d ^{(2)}_1}}\left( d_1\otimes d_2 \right) \right) \right] \nonumber \\&\quad + \frac{1}{\sqrt{N}} \frac{1}{ \eta ^{3/2}} E\left[ \tau ^\top d_1\tau ^\top d_2 \left( { {\tau ^\top d^{(1)}_2}} d_1-\frac{1}{2} \tau ^\top {\overline{d ^{(2)}_1}}\left( d_2\otimes d_1\right) \right) \right] + O\left( N^{-3/2}\right) \nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}}\left\{ \eta \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2}\eta \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right\} \nonumber \\&\quad + \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ 2 \tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - \tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) \right\} + O\left( N^{-3/2}\right) \nonumber \\ \end{aligned}$$
(7.11)

We can therefore write the third cumulant as

$$\begin{aligned} E\left[ \left( \xi - E\left[ \xi \right] \right) ^3 \right]&= E\left[ \xi ^3 \right] -3 E\left[ \xi ^2 \right] E\left[ \xi \right] +2 \left( E\left[ \xi \right] \right) ^3\nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ - E\left[ (\tau ^\top d_i)^3\right] + 3\left( \eta \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2}\eta \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right) \right\} \nonumber \\&+ \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} 3 \left\{ 2\tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - \tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) \right\} \nonumber \\&-3 {1\over \sqrt{N}}{{1\over \sqrt{\eta }} }\left\{ \tau ^\top E\left[ {d_1^{(1)} }d_1\right] - {1 \over 2} \tau ^\top \overline{d_1^{(2)}} \text {Vec}\,\mathcal{V} \right\} + O\left( N^{-3/2}\right) \nonumber \\&= \frac{1}{\sqrt{N}} \frac{1}{\eta ^{3/2}} \left\{ 6 \tau ^\top E[ d_1 \tau ^\top {d_1^{(1)} }]\mathcal{V} \tau - 3\tau ^\top {\overline{d ^{(2)}_1}} (\mathcal{V}\tau \otimes \mathcal{V}\tau ) - E[(\tau ^\top d_1)^3] \right\} \nonumber \\&+\,\, O\left( N^{-3/2}\right) . \end{aligned}$$
(7.12)

Proof to Proposition 1

We note that

$$\begin{aligned} K^{[1]}( t(x) ) -\widehat{K}^{[1]}(\widehat{t})= x-x= 0 \end{aligned}$$
(7.13)

so that

$$\begin{aligned}&K^{[1]}( \widehat{t}) -\widehat{K}^{[1]}(\widehat{t}) = (t-\widehat{t}) K^{[2]}( \widehat{t})\end{aligned}$$
(7.14)
$$\begin{aligned}&(t- \widehat{t}) = \frac{K^{[1]}( \widehat{t}) -\widehat{K}^{[1]}(\widehat{t})}{ K^{[2]}( \widehat{t})} = o(N^{-1/2}) \end{aligned}$$
(7.15)

Similarly,

$$\begin{aligned} K^{[j]} ( t(x)) -\widehat{K} ^{[j]}(\widehat{t})&= K^{[j]} ( \widehat{t}) -\widehat{K}^{[j]} (\widehat{t}) + (t(x) -\widehat{t}) K^{[j+1]}( \bar{t})\nonumber \\&= o(N^{-1/2}),\qquad j=0,1,2 \end{aligned}$$
(7.16)

where \(\bar{t}\) is a mean value.

$$\begin{aligned} f(x;K) -\widehat{f}(x)&= \frac{1}{\sqrt{2\pi }} \left( \frac{1}{\sqrt{ K^{[2]}( {t})}}\exp \left\{ K( t(x) ) - {t}(x)x\right\} \right. \nonumber \\&\left. - \frac{1}{\sqrt{ \widehat{K}^{[2]}(\widehat{t})}}\exp \left\{ K(\widehat{t}) - \widehat{t}x\right\} \right) \nonumber \\&= o\left( N^{-1/2}\right) . \end{aligned}$$
(7.17)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kundhi, G., Rilstone, P. Saddlepoint expansions for GEL estimators. Stat Methods Appl 24, 1–24 (2015). https://doi.org/10.1007/s10260-014-0277-4

Download citation

Keywords

  • Higher order asymptotics
  • Edgeworth expansions
  • Saddlepoint expansions
  • Generalized empirical likelihood
  • Generalized method of moments