Edgeworth Expansions and the Bootstrap

Abstract

This chapter outlines the proof of the validity of a properly formulated version of the formal Edgeworth expansion, and derives from it the precise asymptotic rate of the coverage error of Efron’s bootstrap. A number of other applications of Edgeworth expansions are outlined.

Appendices

Appendix: Approximate Moments and Cumulants of W _n

Write $H(\overline{Z}) - H(\mu ) = G(\overline{Z}) + O_{p}(n^{-3/2})$, where

$$\displaystyle\begin{array}{rcl} G(\overline{Z})& =& (\overline{Z}-\mu ) \cdot \mathrm{ grad}\,H(\mu ) + \frac{1} {2!}\sum _{1\leq i_{1},i_{2}\leq k}D_{i_{1}i_{2}}(\overline{Z}^{(i_{1})} -\mu ^{(i_{1})})(\overline{Z}^{(i_{2})} -\mu ^{(i_{2})}) {}\\ & & + \frac{1} {3!}\sum _{1\leq i_{1},i_{2},i_{3}\leq k}D_{i_{1}i_{2}i_{3}}(\overline{Z}^{(i_{1})} -\mu ^{(i_{1})})(\overline{Z}^{(i_{2})} -\mu ^{(i_{2})})(\overline{Z}^{(i_{3})} -\mu ^{(i_{3})}). {}\\ & & \quad [(\overline{Z}-\mu ) \cdot \mathrm{ grad}\,H(\mu ) =\sum _{i}(D_{i}H)(\mu )(\overline{Z}^{(i)} -\mu ^{(i)})]. {}\\ \end{array}$$

Notation $(D_{i}H)(z) = (\partial H/\partial z^{(i)})(z)$, $(D_{i_{1}i_{2}}H)(z) = (D_{i_{1}}D_{i_{2}}H)(z)$, D _i  = (D _i H)(μ), $D_{i_{1}i_{2}} = (D_{i_{1}i_{2}}H)(\mu )$, $D_{i_{1}i_{2}i_{3}} = (D_{i_{1}i_{2}i_{3}}H)(\mu )$, etc., $\sigma _{i_{1}i_{2}} = E(Z_{j}^{(i_{1})} -\mu ^{(i_{1})})(Z_{j}^{(i_{2})} -\mu ^{(i_{2})}) =\mu _{i_{ 1}i_{2}}$, $\mu _{i_{1}i_{2}i_{3}} = E(Z_{j}^{(i_{1})} -\mu ^{(i_{1})}) \cdot (Z_{j}^{(i_{2})} -\mu ^{(i_{2})})(Z_{n}^{(i_{3})} -\mu ^{(i_{3})})$, etc., $m_{r,n}:= EG(\overline{Z})^{r}$, $\mu _{r,n} = E(\sqrt{n}G(\overline{Z}))^{r} = n^{r/2}m_{r,n}$. We will compute μ _r, n up to $O(n^{-3/2})\ (1 \leq r \leq 6)$.

$$\displaystyle\begin{array}{rcl} m_{1,n}& =& \frac{a_{1}} {n} + O(n^{-2}),\ a_{ 1} = \frac{1} {2!}\sum _{i_{1},i_{2}}D_{i_{1}i_{2}}\sigma _{i_{1}i_{2}},{}\end{array}$$

(11.55)

$$\displaystyle\begin{array}{rcl} m_{2,n}& =& \frac{b_{1}} {n} + \frac{b_{2}} {n^{2}} + O(n^{-3}), \\ b_{1}& =& \sum _{i_{1},i_{2}}D_{i_{1}} \cdot D_{i_{2}}\sigma _{i_{1}i_{2}} = E((Z_{j}-\mu )\mathrm{grad}\,H(\mu ))^{2}, \\ b_{2}& =& \sum _{i_{1},i_{2},i_{3}}D_{i_{1}}D_{i_{2}i_{3}}\mu _{i_{1}i_{2}i_{3}} \\ & & \quad + \frac{1} {3}\sum _{i_{1},i_{2},i_{3},i_{4}}D_{i_{1}}D_{i_{2}i_{3}i_{4}}(\sigma _{i_{1}i_{2}}\sigma _{i_{3}i_{4}} +\sigma _{i_{1}i_{3}}\sigma _{i_{2}i_{4}} +\sigma _{i_{1}i_{4}}\sigma _{i_{2}i_{3}}) \\ & & \qquad + \frac{1} {4}\sum _{i_{1},i_{2},i_{3},i_{4}}D_{i_{1}i_{2}}D_{i_{3}i_{4}}(\sigma _{i_{1}i_{2}}\sigma _{i_{3}i_{4}} +\sigma _{i_{1}i_{3}}\sigma _{i_{2}i_{4}} +\sigma _{i_{1}i_{4}}\sigma _{i_{2}i_{3}}),{}\end{array}$$

(11.56)

$$\displaystyle\begin{array}{rcl} m_{3,n}& =& \frac{c_{1}} {n^{2}} + O(n^{-3}), \\ c_{1}& =& \sum _{i_{1},i_{2},i_{3}}D_{i_{1}}D_{i_{2}}D_{i_{3}}\mu _{i_{1}i_{2}i_{3}} \\ & & \quad + \frac{3} {2}\sum _{i_{1},i_{2},i_{3},i_{4}}D_{i_{1}}D_{i_{2}}D_{i_{3}i_{4}}(\sigma _{i_{1}i_{2}}\sigma _{i_{3}i_{4}} +\sigma _{i_{1}i_{3}}\sigma _{i_{2}i_{4}} +\sigma _{i_{1}i_{4}}\sigma _{i_{2}i_{3}}),{}\end{array}$$

(11.57)

$$\displaystyle\begin{array}{rcl} m_{4,n}& =& \frac{d_{1}} {n^{2}} + \frac{d_{2}} {n^{3}} + O(n^{-4}),\ \text{where} \\ d_{1}& =& 3b_{1}^{2},\ \text{and} \\ d_{2}& =& E[(Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu )]^{4} - 3b_{ 1}^{2} \\ & & \quad + 2\sum _{i_{1},i_{2}}D_{i_{1}i_{2}}\bigg[E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{3}\sigma _{ i_{1}i_{2}} \\ & & \quad \ \ + 3E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2} \\ & & \quad \ \ \ \cdot E\{(Z_{1}^{(i_{1})} -\mu ^{(i_{1})})(Z_{ 1}^{(i_{2})} -\mu ^{(i_{2})})((Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))\} \\ & & \quad \ + 3E\{(\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i)} -\mu ^{(i_{1})})\} \\ & & \quad \ \ \ \ \cdot E\{(\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))^{2} \cdot (Z_{ 1}^{(i_{2})} -\mu ^{(i_{2})})\} \\ & & \qquad \ + 3E\{(\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})})\} \\ & & \qquad \ \ \ \cdot E\{(\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))^{2}(Z_{ 1}^{(i_{1})} -\mu ^{(i_{1})})\}\bigg] \\ & & \qquad + \frac{2} {3}\sum _{i_{1},i_{2},i_{3}}D_{i_{1}i_{2}i_{3}}\bigg[3E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2} \\ & & \qquad \quad \ \ \cdot \bigg\{ E((Z_{1}^{(i_{1})} -\mu ^{(i_{1})})(Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu )))\sigma _{i_{2}i_{3}} \\ & & \qquad \quad \ \ + E((Z_{1}^{(i_{2})} -\mu ^{(i_{2})})((Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu )))\sigma _{i_{1}i_{3}} \\ & & \qquad \quad \ \ + E((Z_{1}^{(i_{3})} -\mu ^{(i_{3})})((Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu )))\sigma _{i_{1}i_{2}}\bigg\} \\ & & \qquad \quad \ \ + 6E((Z_{1}^{(i_{1})} -\mu ^{(i_{1})})((Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))) \\ & & \qquad \quad \ \ \cdot E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})}) \\ & & \qquad \quad \ \ \ \cdot E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))(Z_{1}^{(i_{3})} -\mu ^{(i_{3})})\bigg] \\ & & \qquad \quad + \frac{3} {2}\sum _{i_{1},i_{2},i_{3},i_{4}}D_{i_{1}i_{2}}D_{i_{3}i_{4}}\bigg[E(\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))^{2} \\ & & \qquad \qquad \cdot \{\sigma _{i_{1}i_{2}}\sigma _{i_{3}i_{4}} +\sigma _{i_{1}i_{3}}\sigma _{i_{2}i_{4}} +\sigma _{i_{1}i_{4}}\sigma _{i_{2}i_{3}}\} \\ & & \qquad \qquad \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{1})} -\mu ^{(i_{1})})) \\ & & \qquad \qquad \ \ \ \cdot \bigg\{ E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})}))\sigma _{ i_{3}i_{4}} \\ & & \qquad \qquad \ \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{3})} -\mu ^{(i_{3})}))\sigma _{ i_{2}i_{4}} \\ & & \qquad \qquad \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{4})} -\mu ^{(i_{4})})\sigma _{ i_{2}i_{3}}\bigg\} \\ & & \qquad \qquad \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})})) \\ & & \qquad \qquad \ \ \ \ \ \ \cdot \bigg\{ E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{1})} -\mu ^{(i_{1})}))\sigma _{ i_{3}i_{4}} \\ & & \qquad \qquad \ \ \ \ \ \ \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{3})} -\mu ^{(i_{3})}))\sigma _{ i_{1}i_{4}} \\ & & \qquad \qquad \qquad \ \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}^{(i_{4})} -\mu ^{(i_{4})}))\sigma _{ i_{1}i_{3}}\bigg\} \\ & & \qquad \qquad + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{3})} -\mu ^{(i_{3})})) \\ & & \qquad \qquad \cdot \bigg\{ E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{1})} -\mu ^{(i_{1})}))\sigma _{ i_{2}i_{4}} \\ & & \qquad \qquad + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})}))\sigma _{ i_{1}i_{4}} \\ & & \qquad \qquad + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{4})} -\mu ^{(i_{4})}))\sigma _{ i_{1}i_{2}}\bigg\} \\ & & \qquad \qquad + E((\mathrm{grad}\,H(\mu ) \cdot Z_{1}-\mu ))(Z_{1}^{(i_{4})} -\mu ^{(i_{4})})) \\ & & \qquad \qquad \qquad \ \cdot \bigg\{ E((\mathrm{grad}\,H(\mu ) \cdot Z_{1}-\mu ))(Z_{1}^{(i_{1})} -\mu ^{(i_{1})}))\sigma _{ i_{2}i_{3}} \\ & & \qquad \qquad \qquad \ \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}-\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})}))\sigma _{ i_{1}i_{3}} \\ & & \qquad \qquad \qquad \ \ + E((\mathrm{grad}\,H(\mu ) \cdot (Z_{1}^{(i_{3})} -\mu ^{(i_{3})}))\sigma _{ i_{1}i_{2}}\bigg\}\bigg], {}\end{array}$$

(11.58)

$$\displaystyle\begin{array}{rcl} m_{5,n}& =& \frac{e_{1}} {n^{3}} + O(n^{-4}), \\ e_{1}& =& 10E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{3} \cdot E((Z_{ 1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2} \\ & & \quad + \frac{5} {2}\sum _{i_{1},i_{2}}D_{i_{1}i_{2}}E(Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2} \\ & & \qquad \cdot \bigg\{ 3\sigma _{i_{1}i_{2}}(E(Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2} \\ & & \qquad \quad + 12E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))(Z_{1}^{(i_{1})} -\mu ^{(i_{1})})) \\ & & \qquad \qquad \cdot E((Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))(Z_{1}^{(i_{2})} -\mu ^{(i_{2})}))\bigg\}, {}\end{array}$$

(11.59)

$$\displaystyle\begin{array}{rcl} m_{6,n}& =& \frac{f_{1}} {n^{3}} + O(n^{-4}), \\ f_{1}& =& 15[E(Z_{1}-\mu ) \cdot \mathrm{ grad}\,H(\mu ))^{2}]^{3}.{}\end{array}$$

(11.60)

“Approximate” cumulants of W _n are then given by

$$\displaystyle\begin{array}{rcl} k_{1,n}& =& \mu _{1,n} = \sqrt{n}\,m_{1,n} = n^{-1/2}a_{ 1} + O(n^{-3/2}) \\ & =& \frac{n^{-1/2}} {2} \sum _{i_{1},i_{2}}D_{i_{1}i_{2}}\sigma _{i_{1}i_{2}} + O(n^{-3/2}) \\ k_{2,n}& =& \mu _{2,n} -\mu _{1,n}^{2} = n(m_{ 2,n} - m_{1,n}^{2}) \\ & =& b_{1} + \frac{b_{2} - a_{1}^{2}} {n} + O(n^{-2}), \\ k_{3,n}& =& \mu _{3,n} - 3\mu _{2,n}\mu _{1,n} + 2\mu _{1,n}^{3} = n^{-1/2}(c_{ 1} - 3b_{1}a_{1}) + O(n^{-3/2}), \\ k_{4,n}& =& \mu _{4,n} - 4\mu _{3,n}\mu _{1,n} - 3\mu _{2,n}^{2} + 12\mu _{ 2,n}\mu _{1,n}^{2} - 6\mu _{ 1,n}^{4} \\ & =& d_{1} + \frac{d_{2}} {n} -\frac{4a_{1}c_{1}} {n} - 3\left (b_{1}^{2} + \frac{2b_{1}b_{2}} {n} \right ) + 12\frac{a_{1}^{2}b_{1}} {n} + O(n^{-2}) \\ & =& n^{-1}(d_{ 2} - 4a_{1}c_{1} - 6b_{1}b_{2} + 12a_{1}^{2}b_{ 1}) + O(n^{-2}). {}\end{array}$$

(11.61)

As an example for Student’s t,

$$\displaystyle\begin{array}{rcl} k_{1,n}& =& -\frac{1} {2}\mu _{3}n^{-1/2} + O(n^{-3/2}), \\ k_{2,n}& =& 1 + n^{-1}(2\mu _{ 3}^{2} + 3) - n^{-1}\left (\frac{1} {4}\mu _{3}^{2}\right ) + O(n^{-2}) \\ & =& 1 + n^{-1}\left (\frac{7} {4}\mu _{3}^{2} + 3\right ) + O(n^{-2}) \\ k_{3,n}& =& n^{-1/2}\left (-\frac{7} {2}\mu _{3} + \frac{3} {2}\mu _{3}\right ) + O(n^{-3/2}) = -2n^{-1/2}\mu _{ 3} + O(n^{-3/2}) \\ k_{4,n}& =& n^{-1}[-2\mu _{ 4} + 28\mu _{3}^{2} + 30 - 7\mu _{ 3}^{2} - 6(2\mu _{ 3}^{2} + 3) + 3\mu _{ 3}^{2}] + O(n^{-2}) \\ & =& n^{-1}(-2\mu _{ 4} + 12\mu _{3}^{2} + 12) + O(n^{-2}) \\ & =& -2n^{-1}(\mu _{ 4} - 6\mu _{3}^{2} - 6) + O(n^{-2}). {}\end{array}$$

(11.62)

Exercises for Chap. 11

Ex. 11.1. Derive (a) the Edgeworth expansion for the distribution function of the (nonparametric) Student’s t, using (11.62), and under appropriate conditions, and (b) prove the analog of (11.49) for the coverage error of the bootstrap approximation for a symmetric confidence interval for the mean based on t.