Asymptotic Variance of Test Statistics in the ML and QML Frameworks

Abstract

In this study, we consider the test statistics that can be written as the sample average of data and derive their limiting distributions under the maximum likelihood (ML) and the quasi-maximum likelihood (QML) frameworks. We first generalize the asymptotic variance formula suggested in Pierce (Ann Stat 10(2):475–478, 1982) in the ML framework and illustrate its applications through some well-known test statistics: (1) the skewness statistic, (2) the kurtosis statistic, (3) the Cox statistic, (4) the information matrix test statistic, and (5) the Durbin’s h-statistic. We next provide a similar result in the QML setting and illustrate its applications by providing two examples. Illustrations show the simplicity and the effectiveness of our results for the asymptotic variance of test statistics, and therefore, they are recommended for practical applications.

Appendices

Appendix

A Lemmas

In this section, we provide two useful lemmas that are required in the proofs of Propositions 1 and 2. The first lemma gives the conditions for the uniform law of large numbers for functions of mixing sequences. The second lemma shows how a consistent estimator of a term that obeys the uniform law of large numbers can be formulated.

Lemma 1

Assume that Assumption 1holds, and let $\{V_t\}$ be a mixing process with $\phi _m$ of size $-r/(2r-2)$, $r\ge 2$, or $\alpha _m$ of size $-r/(r-2)$, $r>2$. Suppose that

• (1) $\log f_{nt}(Y^t_n,\theta )$ is Lipschitz continuous on $\Theta$, where $\Theta$ is compact set.

• (2) $\log f_{nt}(Y^t_n,\theta )$ is r-dominated on $\Theta$ uniformly in t and n.

• (3) $\{\log f_{nt}(Y^t_n,\theta )\}$ is near epoch dependent on $\{V_t\}$ of size $-1/2$ uniformly on $(\Theta ,\varrho )$, where $\varrho$ is a metric on $\Theta$.

Then, (a) $\mathbb {E}(\log f_{nt}(Y^t_n,\theta ))$ is continuous on $\Theta$ uniformly in n and t, and (b) $\log f_{nt}(Y^t_n,\theta ) -\mathbb {E}\left( \log f_{nt}(Y^t_n,\theta )\right) \xrightarrow {}0$ a.s. -$\text {P}_0$ uniformly on $\Theta$.

Proof

See Gallant and White [12, Theorem 3.18]. $\square$

Lemma 2

Let $\{Q_n:\Omega \times \Theta \xrightarrow {}\mathbb {R}\}$ be a sequence of continuous function on $\Theta$ a.s.-$P_0$, and let $\{\hat{\theta }_n:\Omega \xrightarrow {}\Theta \}$ be a sequence satisfying $\hat{\theta }_n-\theta ^*_n\xrightarrow {}0$ a.s.-$P_0$. Suppose that $\sup _{\theta \in \Theta }\left| Q_n(\cdot ,\theta )-\bar{Q}_n(\theta )\right| \xrightarrow {}0$ a.s.-$P_0$, where $\{\bar{Q}_n:\Theta \xrightarrow {}\mathbb {R}\}$ is continuous on $\Theta$ uniformly in n. Then,

$$\begin{aligned} Q_n(\cdot ,\hat{\theta }_n)-\bar{Q}_n(\theta ^*_n)\xrightarrow {}0\quad {\textit{ a.s.-}}P_0. \end{aligned}$$

Proof

See White [36, Corollary 3.8]. $\square$

B Proofs of Propositions

1.1 B.1 Proof of Proposition 1

Define the following vector

$$\begin{aligned} \xi _{nt}(Y^t_n, \theta ,\rho )= \begin{pmatrix} \frac{1}{n}\frac{\partial \log f_{nt}(Y^t_n,\theta )}{\partial \theta }\\ \frac{1}{n}(\rho _{nt}(Y^t_n,\theta )-\rho ) \end{pmatrix}. \end{aligned}$$

(B.1)

Under assumption that $\hat{\theta }_n-\theta _0\xrightarrow {}0$ a.s. $\text {P}_0$, we have $\sum _{t=1}^n\xi _{nt}(Y^t_n, \hat{\theta }_n,\hat{\rho }_n)=0$ a.s. $\text {P}_0$ by Lemmas 1 and 2. Also note that

$$\begin{aligned} \mathrm {Var}\left( \sqrt{n}\sum _{t=1}^n\xi _{nt}(Y^t_n, \theta _0,\rho _0)\right) = \begin{pmatrix} \mathcal {A}_n(\theta _0)&{}\mathcal {P}_n(\theta _0,\rho _0)\\ \mathcal {P}^{'}_n(\theta _0,\rho _0)&{}\mathcal {C}_n(\theta _0,\rho _0) \end{pmatrix}, \end{aligned}$$

(B.2)

where $\mathcal {P}_n(\theta _0,\rho _0)=\mathbb {E}\left( \left( \frac{1}{\sqrt{n}}\sum _{t=1}^n\frac{\partial \log f_{nt}(Y^t_n,\theta _0)}{\partial \theta }\right) \times \left( \frac{1}{\sqrt{n}}\sum _{t=1}^n(\rho _{nt}(Y^t_n,\theta _0)-\rho _0)\right) ^{'}\right)$ and $\mathcal {C}(\theta _0,\rho _0)=\mathrm {Var}\left( \frac{1}{\sqrt{n}}\sum _{t=1}^n(\rho _{nt}(Y^t_n,\,\theta _0)-\rho _0)\right)$. Let $\nabla _\delta$ be the gradient with respect to $\delta$. Then, taking a mean value expansion of $\xi _{nt}(Y^t_n, \hat{\theta }_n,\hat{\rho }_n)$ around $\theta _0$ and $\rho _0$ gives

$$\begin{aligned} \sum _{t=1}^n\sqrt{n}\xi _{nt}(Y^t_n, \theta _0,\rho _0)=\left( -\sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \tilde{\theta }_n,\tilde{\rho }_n)\right) \begin{pmatrix} \sqrt{n}(\hat{\theta }_n-\theta _0)\\ \sqrt{n}\left( T_n(Y^n_n,\hat{\theta }_n)-\rho _0\right) \end{pmatrix} \quad a.s. \text {-P}_0, \end{aligned}$$

(B.3)

where $\tilde{\theta }_n$ and $\tilde{\rho }_n$ are the mean values. Note that

$$\begin{aligned} \mathbb {E}\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \theta _0,\rho _0)\right) =\mathbb {E}\begin{pmatrix} n^{-1}\sum _{t=1}^n\frac{\partial ^2\log f_{nt}(Y^t_n,\theta _0)}{\partial \theta \partial \theta ^{'}}&{}0\\ n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\theta _0)}{\partial \theta ^{'}}&{}-I_q \end{pmatrix} = \begin{pmatrix} -\mathcal {A}_n(\theta _0)&{}0\\ \mathcal {D}_n(\theta _0)&{}-I_q \end{pmatrix}, \end{aligned}$$

(B.4)

where $\mathcal {D}_n(\theta _0)=\mathbb {E}\left( n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\,\theta _0)}{\partial \theta ^{'}}\right)$ and $I_q$ is the $q\times q$ identity matrix. Our Lemmas 1 and 2 ensure that

$$\begin{aligned} -\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \tilde{\theta }_n,\tilde{\rho }_n)\right) -\begin{pmatrix} -\mathcal {A}_n(\theta _0)&{}0\\ \mathcal {D}_n(\theta _0)&{}-I_q \end{pmatrix} \xrightarrow {}0\quad a.s.\text {P}_0. \end{aligned}$$

(B.5)

Since $\mathcal {A}_n(\theta _0)$ is uniformly non-singular, it follows from (B.5) that $\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \tilde{\theta }_n,\tilde{\rho }_n)\right)$ is non-singular $a.s.\text {P}_0$. Thus,

$$\begin{aligned} \begin{pmatrix} \sqrt{n}(\hat{\theta }_n-\theta _0)\\ \sqrt{n}\left( T_n(Y^n_n,\hat{\theta }_n)-\rho _0\right) \end{pmatrix} =-\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \tilde{\theta }_n,\tilde{\rho }_n)\right) ^{-1}\sum _{t=1}^n\sqrt{n}\xi _{nt}(Y^t_n, \theta _0,\rho _0)\quad a.s.\text {-P}_0. \end{aligned}$$

(B.6)

Our stated assumptions ensure the conditions of the central limit theorem for near epoch-dependent functions of a mixing process (White [36], Theorem A.3.7, p. 358). Thus, we have

$$\begin{aligned} \sum _{t=1}^n\sqrt{n}\xi _{nt}(Y^t_n, \theta _0,\rho _0){\mathop {\sim }\limits ^{A}} N\left[ 0,\, \begin{pmatrix} \mathcal {A}_n(\theta _0)&{}\mathcal {P}_n(\theta _0,\rho _0)\\ \mathcal {P}^{'}_n(\theta _0,\rho _0)&{}\mathcal {C}_n(\theta _0,\rho _0) \end{pmatrix} \right] . \end{aligned}$$

(B.7)

Then, using (B.7) in (B.6), we obtain

$$\begin{aligned} \begin{pmatrix} \sqrt{n}(\hat{\theta }_n-\theta _0)\\ \sqrt{n}\left( T_n(Y^n_n,\hat{\theta }_n)-\rho _0\right) \end{pmatrix} {\mathop {\sim }\limits ^{A}} N\left[ 0,\, \begin{pmatrix} \mathcal {A}^{-1}_n(\theta _0)&{}\mathcal {V}^{'}_n(\theta _0,\rho _0)\\ \mathcal {V}_n(\theta _0,\rho _0)&{}\mathcal {S}_n(\theta _0,\rho _0) \end{pmatrix} \right] , \end{aligned}$$

(B.8)

where

$$\begin{aligned}&\begin{pmatrix} \mathcal {A}^{-1}_n(\theta _0)&{}\mathcal {V}^{'}_n(\theta _0,\rho _0)\\ \mathcal {V}_n(\theta _0,\rho _0)&{}\mathcal {S}_n(\theta _0,\rho _0) \end{pmatrix} \\&\quad = \begin{pmatrix} -\mathcal {A}_n(\theta _0)&{}0\\ \mathcal {D}_n(\theta _0)&{}-I_q \end{pmatrix}^{-1} \begin{pmatrix} \mathcal {A}_n(\theta _0)&{}\mathcal {P}_n(\theta _0,\rho _0)\\ \mathcal {P}^{'}_n(\theta _0,\rho _0)&{}\mathcal {C}_n(\theta _0,\rho _0) \end{pmatrix} \begin{pmatrix} -\mathcal {A}_n(\theta _0)&{}\mathcal {D}^{'}_n(\theta _0)\\ 0&{}-I_q \end{pmatrix}^{-1}. \end{aligned}$$

An application of the inverse partitioned matrix formula gives

$$\begin{aligned}&\begin{pmatrix} -\mathcal {A}_n(\theta _0)&{}0\\ \mathcal {D}_n(\theta _0)&{}-I_q \end{pmatrix}^{-1} =\begin{pmatrix} -\mathcal {A}^{-1}_n(\theta _0)&{}0\\ -\mathcal {D}_n(\theta _0)\mathcal {A}^{-1}_n(\theta _0)&{}-I_q \end{pmatrix}. \end{aligned}$$

(B.9)

Then, using (B.9), we can express $\mathcal {V}_n(\theta _0,\rho _0)$ and $\mathcal {S}_n(\theta _0,\rho _0)$ in (B.8) as

$$\begin{aligned}&\mathcal {V}_n(\theta _0,\rho _0)=\mathcal {D}_n(\theta _0)\mathcal {A}^{-1}_n(\theta _0)+\mathcal {P}^{'}_n(\theta _0,\rho _0)\mathcal {A}^{-1}_n(\theta _0), \end{aligned}$$

(B.10)

$$\begin{aligned}\mathcal {S}_n(\theta _0,\rho _0)=&\,\mathcal {C}_n(\theta _0,\rho _0)+\mathcal {D}_n(\theta _0)\mathcal {A}^{-1}_n(\theta _0)\mathcal {D}^{'}_n(\theta _0) +\mathcal {P}^{'}_n(\theta _0,\rho _0) \mathcal {A}^{-1}_n(\theta _0)\mathcal {D}^{'}_n(\theta _0)\nonumber \\& +\,\mathcal {D}_n(\theta _0)\mathcal {A}^{-1}_n(\theta _0)\mathcal {P}_n(\theta _0,\rho _0). \end{aligned}$$

(B.11)

Next, we show how $\mathcal {S}_n(\theta _0,\rho _0)$ is simplified under Assumption 11(2). The assumption that $\mathbb {E}(\rho _{nt}(Y^t_n,\theta _0))$ is independent of $\theta _0$ implies that

$$\begin{aligned}\frac{\partial \mathbb {E}\left( T(Y^n_n,\theta )\right) }{\partial \theta ^{'}}\big |_{\theta _0}&=n^{-1}\sum _{t=1}^n\frac{\partial \mathbb {E}(\rho _{nt}(Y^t_n,\theta ))}{\partial \theta ^{'}}\big |_{\theta _0}\nonumber \\&\quad =\int \left( n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\theta )}{\partial \theta ^{'}}\bigg |_{\theta _0}\right) \times f_n(Y^n_n,\theta _0)\text {d}\mu ^n\nonumber \\&\quad +\,\int \left( \frac{1}{\sqrt{n}} \sum _{t=1}^n \rho _{nt}(Y^t_n,\theta )\right) \left( \frac{1}{\sqrt{n}}\frac{\partial \log f_n(Y^n_n,\theta )}{\partial \theta }\bigg |_{\theta _0}\right) ^{'} f_n(Y^n_n,\theta _0)\text {d}\mu ^n=0. \end{aligned}$$

(B.12)

Since $\mathbb {E}\left( \frac{\partial \log f_n(Y^n_n,\theta )}{\partial \theta }\big |_{\theta _0}\right) =0$ by Assumption 7, (B.12) can be expressed as

$$\begin{aligned}&\int \left( n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\theta )}{\partial \theta ^{'}}\bigg |_{\theta _0}\right) \times f_n(Y^t_n,\theta _0)\text {d}\mu ^n\nonumber \\&\quad +\,\int \left( \frac{1}{\sqrt{n}} \sum _{t=1}^n \rho _{nt}(Y^t_n,\theta )-\rho _0\right) \left( \frac{1}{\sqrt{n}}\sum _{t=1}^n\frac{\partial \log f_{nt}(Y^t_n,\theta )}{\partial \theta }\bigg |_{\theta _0}\right) ^{'} f_n(Y^n_n,\theta _0)\text {d}\mu ^n=0, \end{aligned}$$

(B.13)

which implies that

$$\begin{aligned} \mathcal {P}^{'}_n(\theta _0,\rho _0)=-\mathcal {D}_n(\theta _0). \end{aligned}$$

(B.14)

Using (B.14) in (B.10) and (B.11) for $\mathcal {V}_n(\theta _0,\rho _0)$ and $\mathcal {S}_n(\theta _0,\rho _0)$, respectively, yields the desired results.

1.2 B.2 Proof of Proposition 2

The proof is similar to that of Proposition 1. Consider the following vector

$$\begin{aligned} \xi _{nt}(Y^t_n, \theta ,\rho )= \begin{pmatrix} \frac{1}{n}\frac{\partial \log f_{nt}(Y^t_n,\theta )}{\partial \theta }\\ \frac{1}{n}(\rho _{nt}(Y^t_n,\theta )-\rho ) \end{pmatrix}. \end{aligned}$$

(B.15)

Because $\hat{\theta }_n-\theta _\star \xrightarrow {}0$ a.s. $\text {P}_0$, our assumptions ensure that $\sum _{t=1}^n\xi _{nt}(Y^t_n, \hat{\theta }_n,\hat{\rho }_n)=0$ a.s. -$\text {P}_0$ by Lemmas 1 and 2. Here, it is important to note that

$$\begin{aligned} \mathrm {Var}\left( \sqrt{n}\sum _{t=1}^n\xi _{nt}(Y^t_n, \theta _\star ,\rho _\star )\right) = \begin{pmatrix} \mathcal {B}_n(\theta _\star )&{}\mathcal {P}_n(\theta _\star ,\rho _\star )\\ \mathcal {P}^{'}_n(\theta _\star ,\rho _\star )&{}\mathcal {C}_n(\theta _\star ,\rho _\star ) \end{pmatrix}, \end{aligned}$$

(B.16)

where $\mathcal {P}_n(\theta _\star ,\rho _\star )=\mathbb {E}\left( \left( \frac{1}{\sqrt{n}}\sum _{t=1}^n\frac{\partial \log f_{nt}(Y^t_n,\theta _\star )}{\partial \theta }\right) \times \left( \frac{1}{\sqrt{n}}\sum _{t=1}^n(\rho _{nt}(Y^t_n,\theta _\star )-\rho _\star )\right) ^{'}\right)$ and $\mathcal {C}(\theta _\star )=\mathrm {Var}\left( \frac{1}{\sqrt{n}}\sum _{t=1}^n(\rho _{nt}(Y^t_n,\,\theta _\star )-\rho _\star )\right)$. Similarly, it can be shown that

$$\begin{aligned} \mathbb {E}\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \theta _\star ,\rho _\star )\right) =\mathbb {E}\begin{pmatrix} n^{-1}\sum _{t=1}^n\frac{\partial ^2\log f_{nt}(Y^t_n,\theta _\star )}{\partial \theta \partial \theta ^{'}}&{}0\\ n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\theta _\star )}{\partial \theta ^{'}}&{}-I_q \end{pmatrix} = \begin{pmatrix} -\mathcal {A}_n(\theta _\star )&{}0\\ \mathcal {D}_n(\theta _\star )&{}-I_q \end{pmatrix}, \end{aligned}$$

(B.17)

where $\mathcal {D}_n(\theta _\star )=\mathbb {E}\left( n^{-1}\sum _{t=1}^n\frac{\partial \rho _{nt}(Y^t_n,\,\theta _\star )}{\partial \theta ^{'}}\right)$ and $I_q$ is the $q\times q$ identity matrix. Also, Lemmas 1 and 2 ensure that

$$\begin{aligned} -\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \hat{\theta }_n,\hat{\rho }_n)\right) - \mathbb {E}\left( \sum _{t=1}^n\nabla _{\theta \rho }\xi _{nt}(Y^t_n, \theta _\star ,\rho _\star )\right) \xrightarrow {}0\quad a.s.\text {P}_0. \end{aligned}$$

(B.18)

Then, following the argument in the proof of Proposition 1, it follows that

$$\begin{aligned} \begin{pmatrix} \sqrt{n}(\hat{\theta }_n-\theta _\star )\\ \sqrt{n}(T_n(Y^n_n,\hat{\theta }_n)-\rho _\star ) \end{pmatrix} {\mathop {\sim }\limits ^{A}} N\left[ 0,\, \begin{pmatrix} \mathcal {A}^{-1}_n(\theta _\star )\mathcal {B}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star )&{}\mathcal {V}^{'}_n(\theta _\star ,\rho _\star )\\ \mathcal {V}_n(\theta _\star ,\rho _\star )&{}\mathcal {S}_n(\theta _\star ,\rho _\star ) \end{pmatrix} \right] , \end{aligned}$$

(B.19)

where

$$\begin{aligned}&\begin{pmatrix} \mathcal {A}^{-1}_n(\theta _\star )\mathcal {B}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star )&{}\mathcal {V}^{'}_n(\theta _\star ,\rho _\star )\\ \mathcal {V}_n(\theta _\star ,\rho _\star )&{}\mathcal {S}_n(\theta _\star ,\rho _\star ) \end{pmatrix} \nonumber \\&\quad = \begin{pmatrix} -\mathcal {A}_n(\theta _\star )&{}0\\ \mathcal {D}_n(\theta _\star )&{}-I_q \end{pmatrix}^{-1} \begin{pmatrix} \mathcal {B}_n(\theta _\star )&{}\mathcal {P}_n(\theta _\star ,\rho _\star )\\ \mathcal {P}^{'}_n(\theta _\star ,\rho _\star )&{}\mathcal {C}_n(\theta _\star ,\rho _\star ) \end{pmatrix} \begin{pmatrix} -\mathcal {A}_n(\theta _\star )&{}\mathcal {D}^{'}_n(\theta _\star )\\ 0&{}-I_q \end{pmatrix}^{-1}. \end{aligned}$$

(B.20)

Using the inverse partitioned matrix formula (see (B.9)) in (B.20), it can be shown that

$$\begin{aligned} \mathcal {V}_n(\theta _\star ,\rho _\star )&=\mathcal {D}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star )\mathcal {B}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star ) +\mathcal {P}^{'}_n(\theta _\star ,\rho _\star )\mathcal {A}^{-1}_n(\theta _\star ), \end{aligned}$$

(B.21)

$$\begin{aligned} \mathcal {S}_n(\theta _\star ,\rho _\star )&=\mathcal {C}_n(\theta _\star ,\rho _\star )+\mathcal {D}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star )\mathcal {B}_n(\theta _\star ) \mathcal {A}^{-1}_n(\theta _\star )\mathcal {D}^{'}_n(\theta _\star )+\mathcal {P}^{'}_n(\theta _\star ,\rho _\star ) \mathcal {A}^{-1}_n(\theta _\star )\mathcal {D}^{'}_n(\theta _\star )\nonumber \\&\quad +\,\mathcal {D}_n(\theta _\star )\mathcal {A}^{-1}_n(\theta _\star )\mathcal {P}_n(\theta _\star ,\rho _\star ). \end{aligned}$$

(B.22)