The Infinitesimal Jackknife and Moment Structure Analysis Using Higher Order Moments

Abstract

Mean corrected higher order sample moments are asymptotically normally distributed. It is shown that both in the literature and popular software the estimates of their asymptotic covariance matrices are incorrect. An introduction to the infinitesimal jackknife is given and it is shown how to use it to correctly estimate the asymptotic covariance matrices of higher order sample moments. Another advantage in using the infinitesimal jackknife is the ease with which it may be used when stacking or sub-setting estimators. The estimates given are used to test the goodness of fit of a non-linear factor analysis model. A computationally accelerated form for infinitesimal jackknife estimates is given.

Appendix

1.1 $\nu _k$ has the form (2)

Note that for any two column vectors $u$ and $v$, $u\otimes v$ is a permutation of $v\otimes u$. Using this repeatedly if $u_1,\ldots ,u_k$ is a permutation of $v_1,\ldots ,v_k$

$$\begin{aligned} u_1\otimes \cdots \otimes u_k=P(v_1\otimes \cdots \otimes v_k), \end{aligned}$$

for some permutation matrix $P$.

Let

$$\begin{aligned} \xi =(x-\mu )\otimes \cdots \otimes (x-\mu ), \end{aligned}$$

where the product is of order $k$. This can be expanded as a sum of $m$ terms of the form

$$\begin{aligned} \xi _i=u_{1}\otimes \cdots \otimes u_{k}, \end{aligned}$$

where each $u_j$ is either $x$ or $\mu$. This can be written in the form

$$\begin{aligned} \xi _i=P(x\otimes \cdots \otimes x\otimes \mu \otimes \cdots \otimes \mu ), \end{aligned}$$

where $P$ is a permutation matrix, the Kronecker product is of order $k$, and the sub-product of the $x$’s is of order $i$.

$$\begin{aligned} E(\xi _i)=P(E(x\otimes \cdots \otimes x)\otimes \mu \otimes \cdots \otimes \mu ). \end{aligned}$$

This is a continuously differentiable function $h_i$ of the expected values of $x$ and $x\otimes \cdots \otimes x$. Thus

$$\begin{aligned} E(\xi _i)=h_i\left( \int t_i(x)\mathrm{{d}}F(x)\right) , \end{aligned}$$

where $t_1(x)=x$ and for $i>1$

$$\begin{aligned} t_i(x)=\left( \begin{array}{c} x\\ x\otimes \cdots \otimes x \end{array}\right) \end{aligned}$$

and the Kronecker product has $i$ terms.

Let

$$\begin{aligned} h\left( \begin{array}{c} u_1\\ \vdots \\ u_m \end{array}\right) =h_1(u_1)+\cdots +h_m(u_m). \end{aligned}$$

Then $h$ is continuously differentiable. Let

$$\begin{aligned} t(x)=\left( \begin{array}{c} t_1(x)\\ \vdots \\ t_m(x) \end{array}\right) . \end{aligned}$$

Using $h$ and $t$

$$\begin{aligned} E(\xi )&= E(\xi _1)+\cdots +E(\xi _m)\\&= h_1\left( \int t_1(x)\mathrm{{d}}F(x)\right) \!+\!\cdots \!+\! h_m \left( \int t_m(x)\mathrm{{d}}F(x)\right) \\&= h\left( \left( \begin{array}{c} \int t_1(x)\mathrm{{d}}F(x))\\ \vdots \\ \int t_m(x)\mathrm{{d}}F(x) \end{array}\right) \right) \\&= h\left( \int \left( \begin{array}{c} t_1(x)\\ \vdots \\ t_m(x) \end{array}\right) \mathrm{{d}}F(x)\right) \\&= h\left( \int t(x)\mathrm{{d}}F(x)\right) . \end{aligned}$$

Thus

$$\begin{aligned} \nu _k=E(\xi )=h\left( \int t(x)\mathrm{{d}}F(x)\right) \end{aligned}$$

has the form (2).

1.2 Computing the Matrix $C_k$

Let $a\in R^m$ and $b\in R^n$. Then $b\otimes a$ is a permutation of $a\otimes b$ and

$$\begin{aligned} P_{mn}(a\otimes b)=b\otimes a, \end{aligned}$$

for some permutation matrix $P_{mn}$. Note that

$$\begin{aligned} P_{m1}a=P_{m1}(1\otimes a)=a\otimes 1=a \end{aligned}$$

Thus

$$\begin{aligned} P_{m1}=I_m \end{aligned}$$

Let $u_1,\ldots ,u_m$ be the coordinate vectors in $R^m$ and $v_1,\ldots ,v_n$ be the coordinate vectors in $R^n$. Then the vectors $u_i\otimes v_j$ with $i=1,\ldots ,m$ and $j=1,\ldots ,n$ are a basis for $R^m\times R^n$. Order this basis lexicographically. Since

$$\begin{aligned} P_{mn}(u_i\otimes v_j) =v_j\otimes u_i \end{aligned}$$

the columns of $P_{mn}$ are $v_j\otimes u_i$ in lexicographic order on $i$ and $j$. All of the components of $v_j\otimes u_i$ are zero except for its $r=(j-1)m+i$ component which is one. Thus the computation of $P_{mn}$ is inexpensive.

In the notation of Section 4 note that the sum of all distinct permutations of

$$\begin{aligned} (x_i-\bar{x})\otimes (x_j-\bar{x})\otimes \cdots \otimes (x_j-\bar{x}) \end{aligned}$$

is equal to the sum on $\ell =0,\ldots ,k-1$ of

$$\begin{aligned} P_{p^{k-\ell } p^\ell }(x_i-\bar{x})\otimes (x_j-\bar{x})\otimes \cdots \otimes (x_j-\bar{x}). \end{aligned}$$

Let

$$\begin{aligned} C_k=\sum \limits _{\ell =0}^{k-1}P_{p^{k-\ell } p^\ell }. \end{aligned}$$

Then

$$\begin{aligned} C_k((x_i-\bar{x})\otimes (x_j-\bar{x})\otimes \cdots \otimes (x_j-\bar{x})) \end{aligned}$$

is the sum of all distinct permutations of $(x_i-\bar{x})\otimes (x_j-\bar{x})\otimes \cdots \otimes (x_j-\bar{x})$ and the mean on $j$ of these is

$$\begin{aligned} C_k((x_i-\bar{x})\otimes q_{k-1}), \end{aligned}$$

as asserted in Section 3.