Inference of nonlinear mixed models for clustered data under moment conditions

Abstract

Two statistical inference problems in nonlinear mixed models (NLMM) are considered under only moment conditions on random effects and random errors. First, higher-order moment estimates of random effects and random errors in NLMM are proposed and they turn out to be strongly consistent. Second, a difference-type test $T_{mDs}$ is developed to test whether some sub-vector of random effects exists or not, which is easy to implement without requiring the Monte Carlo method. Its theoretical properties including the power properties are obtained. Moreover, in the special case of testing the existence of random effects, two kinds of tests are also constructed: the global difference-type test $T_{mDG}$, which is a special case of $T_{mDs}$, and the modified score-type test $ST_{n0}$, which is motivated by $ST_{nru}$ in Russo et al. (TEST 21:519–545, 2012). The simulation study indicates that $T_{mDs}$ is the most powerful. A real data analysis is also conducted to investigate the applicability of the procedures.

Appendices

Appendix 1: Notation

Denote ${\mathbf{S}}_X({\varvec{\beta }}) \equiv \frac{\partial {\mathbf{f}}(\mathbf{X}, {\varvec{\beta }})}{\partial {\varvec{\beta }}^\top }$, ${\mathbf{S}}_X^{(1)}({\varvec{\beta }})=\frac{\partial \left[ \text{ vec }\{{\mathbf{S}}_X({\varvec{\beta }})\}\right] }{\partial {\varvec{\beta }}^\top }$, ${\mathbf{Y}}=\left( {\mathbf{Y}}_1^\top ,\ldots ,\mathbf{Y}_m^\top \right) ^\top$,

$$\begin{aligned} {\mathbf{g}}({\varvec{\beta }})\equiv & {} \{{\mathbf{S}}_X({\varvec{\beta }})\}^\top \{{\mathbf{Y}}-{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})\}, \quad {\varvec{\varepsilon }}=\left( {\varvec{\varepsilon }}_1^\top ,\ldots ,{\varvec{\varepsilon }}_m^\top \right) ^\top , \quad {\mathbf{X}}=\left( {\mathbf{X}}_1^\top ,\ldots ,\mathbf{X}_m^\top \right) ^\top \!\!, \\ {\mathbf{g}}^{(1)}({\varvec{\beta }})\equiv & {} \frac{\partial {{\mathbf{g}}({\varvec{\beta }})}}{ \partial {{\varvec{\beta }}^\top }} ={\mathbf{I}}_p\otimes \{{\mathbf{Y}}-{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})\}^\top {\mathbf{S}}_X^{(1)}({\varvec{\beta }}) -\{{\mathbf{S}}_X({\varvec{\beta }})\}^\top {\mathbf{S}}_X({\varvec{\beta }}), \end{aligned}$$

$$\begin{aligned} \widetilde{\mathbf{Y}}= & {} {\mathbf{P}}_{Z^\bot } {\mathbf{Y}}, \quad \tilde{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }}) = {\mathbf{P}}_{Z^\bot } {\mathbf{f}}({\mathbf{X}},{\varvec{\beta }}), \quad \tilde{{\varvec{\varepsilon }}}={\mathbf{P}}_{Z^\bot } {\varvec{\varepsilon }}, \quad \tilde{\mathbf{S}}_X({\varvec{\beta }})= {\mathbf{P}}_{Z^\bot } {\mathbf{S}}_X({\varvec{\beta }}), \\ \tilde{\mathbf{g}}({\varvec{\beta }})\equiv & {} \{\tilde{\mathbf{S}}_X({\varvec{\beta }})\}^\top \{\tilde{\mathbf{Y}}-\tilde{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})\}, \quad \tilde{\mathbf{f}}_{(1)}({\mathbf{X}},{\varvec{\beta }})={\mathbf{P}}_{Z^{(1)\bot }} \mathbf{f}(X,{\varvec{\beta }}), \\ \tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})\equiv & {} \frac{\partial {\tilde{\mathbf{g}}({\varvec{\beta }})}}{ \partial {{\varvec{\beta }}^\top }} ={\mathbf{I}}_p\otimes \{\tilde{\mathbf{Y}}-\tilde{\mathbf{f}}(\mathbf{X},{\varvec{\beta }})\}^\top \tilde{\mathbf{S}}^{(1)}(\mathbf{X},{\varvec{\beta }}) -\{\tilde{\mathbf{S}}(\mathbf{X},{\varvec{\beta }})\}^\top \tilde{\mathbf{S}}(\mathbf{X},{\varvec{\beta }}), \\ \widetilde{\mathbf{Y}}^{(1)}= & {} {\mathbf{P}}_{Z^{(1)\bot }} Y, \quad \tilde{{\varvec{\varepsilon }}}^{(1)}={\mathbf{P}}_{Z^{(1)\bot }} {\varvec{\varepsilon }}, \quad \tilde{\mathbf{g}}_{(1)}({\varvec{\beta }}) \equiv \{\tilde{\mathbf{S}}_{(1)}({\mathbf{X}},{\varvec{\beta }})\}^\top \{\tilde{\mathbf{Y}}^{(1)}-\tilde{\mathbf{f}}_{(1)}(\mathbf{X},{\varvec{\beta }})\},\\ \tilde{\mathbf{g}}_{(1)}^{(1)}({\varvec{\beta }})\equiv & {} \frac{\partial {\tilde{\mathbf{g}}{(1)}({\varvec{\beta }})}}{ \partial {{\varvec{\beta }}^\top }}\\= & {} {\mathbf{I}}_p\otimes \{\tilde{\mathbf{Y}}^{(1)}-\tilde{\mathbf{f}}_{(1)}({\mathbf{X}},{\varvec{\beta }})\}^\top \tilde{\mathbf{S}}_{(1)}^{(1)}({\mathbf{X}},{\varvec{\beta }}) -\{\tilde{\mathbf{S}}_{(1)}({\mathbf{X}},{\varvec{\beta }})\}^\top \tilde{\mathbf{S}}_{(1)}(\mathbf{X},{\varvec{\beta }}) \end{aligned}$$

where $\tilde{\mathbf{S}}^{(1)}(\mathbf{X},{\varvec{\beta }})$ and $\tilde{\mathbf{S}}^{(1)}_{(1)}(\mathbf{X},{\varvec{\beta }})$ are defined similar to ${\mathbf{S}}_X^{(1)}({\varvec{\beta }})$.

In particular, define the matrix functions of ${\varvec{\beta }}$ as follows:

$$\begin{aligned} {\mathbf{M}}_{m}^{ss}({\varvec{\beta }})= & {} {\mathbf{M}}_{m}^{ssA}({\varvec{\beta }})-{\mathbf{M}}_{m}^{ssF}({\varvec{\beta }}), \quad {\mathbf{M}}_{m}^{ssA}({\varvec{\beta }}) \equiv \left\{ {\mathbf{P}}_{Z^{(1)\bot }}+\widetilde{\mathbf{Sgs1}}_X({\varvec{\beta }})\right\} ^2,\nonumber \\ \widetilde{{\mathbf{Sgs1}}}_X({\varvec{\beta }})= & {} \tilde{{\mathbf{S}}}_{(1)}({\mathbf{X}},{\varvec{\beta }})\left\{ \tilde{{\mathbf{g}}}_{(1)}^{(1)}({\varvec{\beta }})\right\} ^{-1} \left\{ \tilde{\mathbf{S}}_{(1)}(\mathbf{X},{\varvec{\beta }}) \right\} ^\top , \end{aligned}$$

(20)

$$\begin{aligned} {\mathbf{M}}_{m}^{ssF}({\varvec{\beta }})\equiv & {} \left\{ {\mathbf{P}}_{Z^\bot }+\widetilde{\mathbf{Sgs}}_X({\varvec{\beta }})\right\} ^2, \quad \widetilde{\mathbf{Sgs}}_X({\varvec{\beta }})\equiv \tilde{\mathbf{S}}_X({\varvec{\beta }})\left\{ \tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})\right\} ^{-1} \left\{ \tilde{\mathbf{S}}_X({\varvec{\beta }}) \right\} ^\top , \nonumber \\ {\mathbf{M}}_m^{ssG}({\varvec{\beta }})= & {} \left[ {\mathbf{I}}_n+{\mathbf{S}}_X({\varvec{\beta }}) \left\{ {\mathbf{g}}^{(1)}({\varvec{\beta }})\right\} ^{-1} {\mathbf{S}}_X({\varvec{\beta }})^\top \right] ^2 -{\mathbf{M}}_{m}^{ssF}({\varvec{\beta }}). \end{aligned}$$

(21)

Appendix 2: Regularity conditions

In the paper, $n_i\ge 1$ with $n_i's$ bounded and then $n\rightarrow \infty$ means $m\rightarrow \infty$.

C.1. Assume $f(x_{ij},{\varvec{\beta }})$ is a twice continuous differentiable function of ${\varvec{\beta }} \in {\mathscr {B}}$ for each $x_{ij}$ and a measurable function of $x_{ij}$ for each $\beta$ where ${\mathscr {B}}$ is a compact subset of a Euclidean space.

C.2. $Q_{n0}({\varvec{\beta }},{\varvec{\beta }}')\equiv \frac{1}{n} \left\{ \mathbf{f}(\mathbf{X},{\varvec{\beta }})-\mathbf{f}(\mathbf{X},{\varvec{\beta }}')\right\} ^\top \left\{ \mathbf{f}(\mathbf{X},{\varvec{\beta }})-\mathbf{f}(\mathbf{X},{\varvec{\beta }}')\right\}$ converges uniformly to a continuous function $Q_0({\varvec{\beta }},{\varvec{\beta }}')$ and $Q_0({\varvec{\beta }},{\varvec{\beta }}')=0$ if and only if ${\varvec{\beta }}'={\varvec{\beta }}$.

C.3. Assume $\frac{1}{n} \mathbf{G}({\varvec{\beta }})^\top \mathbf{G}({\varvec{\beta }})$ with $\mathbf{G}({\varvec{\beta }})=\left( {\mathbf{f}}({\mathbf{X}},{\varvec{\beta }}),{\mathbf{S}}_{X}({\varvec{\beta }}),{\mathbf{S}}^{(1)}({\mathbf{X}},{\varvec{\beta }})\right)$ and the following items all converge uniformly in ${\varvec{\beta }} \in {\mathscr {B}}$:

• (1) $\lim _{n \rightarrow \infty } \frac{1}{n} {\mathbf{S}}_{X}({\varvec{\beta }})^\top {\mathbf{S}}_{X}({\varvec{\beta }})\equiv {\varvec{\Sigma }}_{S_X}({\varvec{\beta }})$ with ${\varvec{\Sigma }}_{S_X}({\varvec{\beta }})$ being a nonsingular matrix;

• (2) $\lim _{n \rightarrow \infty } \frac{1}{n} {\mathbf{S}}_{X}({\varvec{\beta }})^\top {\mathbf{P}}_{Z^\bot } {\mathbf{S}}_{X}({\varvec{\beta }})\equiv {\varvec{\Sigma }}_{\widetilde{S}_X}(\beta )$ with ${\varvec{\Sigma }}_{\tilde{S}_X}({\varvec{\beta }})$ being a nonsingular matrix;

• (3) $\lim _{n \rightarrow \infty } \frac{1}{n} {\mathbf{S}}_{X}({\varvec{\beta }})^\top {\mathbf{P}}_{Z^{(1)\bot }} {\mathbf{S}}_{X}({\varvec{\beta }})\equiv {\varvec{\Sigma }}_{\widetilde{\mathbf{S}}_X(1)}({\varvec{\beta }})$ with ${\varvec{\Sigma }}_{\tilde{S}_X(1)}({\varvec{\beta }})$ being a nonsingular matrix.

C.4. (1) ${\mathbf{b}}_i's$ are independently and identically distributed (i.i.d.), which are also independent of the i.i.d. $\varepsilon _{ij}'s$; (2) Random effects ${\mathbf{b}}_i's$ and random errors $\varepsilon _{ij}'s$ have the finite fourth-order moments.

C.5. (1) $n_i \ge q_1$ and there exists infinite $n_i$ such that $n_i> q_1$ as $m\rightarrow \infty$;

      (2) $(n_i-q_i)/{\sum _{l=1}^i(n_l-q_l)}=O({1}/{i})$; (3) $\mathrm{rank}\{{\mathbf{S}}_X({\varvec{\beta }})\}$ is bounded and $rz-rz_1\rightarrow \infty$.

C.6. (1) $n^{-1}\sum _{i=1}^m \text{ tr }\left\{ \text{ diag }\left( {\mathbf{P}}_{Z_i^\bot }\right) \right\} ^2$ are bounded away from zero;

      (2) $\sum _{i=1}^\infty \frac{\text{ tr }\{\text{ diag }^2 {\mathbf{P}}_{Z_i^\bot }\}}{\sum _{l=1}^i \text{ tr }\{\text{ diag }^2 {\mathbf{P}}_{Z_l^\bot }\}}< \infty$.

C.7. (1) $m^{-1}\sum _{i=1}^m ({\mathbf{Z}}_i^\top {\mathbf{Z}}_i)^{-1}=O(1)$;

      (2) As $m\rightarrow \infty$, $\text{ tr }\left( {\mathbf{Z}}^\top {\mathbf{Z}} \right) ^{-2}-\left[ \text{ tr }\left\{ ({\mathbf{Z}}^\top {\mathbf{Z}})^{-1}\right\} \right] ^2 /n$ converges to infinity;

      (3) The maximal eigenvalues of ${\mathbf{Z}}_i^\top {\mathbf{Z}}_i$ are bounded.

Remark 2

Conditions C.1–C.3 are the requirements for the nonlinear function, which are similar to those in Vonesh and Carter (1992) and Wu (1981). Condition C.4 is about the independence and the moments of random effects and random errors. Distributional assumptions are not needed for model errors except for the moment conditions in C.4 since our procedures are distribution-free. Conditions C.5–C.7 are conditions of covariates $\mathbf{X}_i,\mathbf{Z}_i$ and $n_i$. Condition C.7 is the requirement for ${\mathbf{Z}}_i$, which is mild. In fact, for the special case with ${\mathbf{Z}}_i=1_{n_0}$, $\mathrm{tr}\left( {\mathbf{Z}}^\top {\mathbf{Z}} \right) ^{-2}-\left[ \mathrm{tr}\left\{ ({\mathbf{Z}}^\top {\mathbf{Z}})^{-1}\right\} \right] ^2 /n= (n_0-1)m/n_0^3 \rightarrow \infty$ as $m\rightarrow \infty$.

Appendix 3: Proofs

Before proving the theorems, we will introduce some lemmas first.

Lemma 1

Let $\{\xi _i\}$ be a sequence of independent random variables with zero means. If $\{a_i\}_{i=1}^k$ is a sequence of real numbers satisfying $a_i\uparrow \infty$ and $\sum _{i=1}^\infty {\mathrm{E}|\xi _i|^\delta }/{a_i^\delta } < \infty$ for some $\delta \in [1,2]$, then $\sum _{i=1}^m \xi _i /a_m \mathop {\rightarrow }\limits ^{a.s.}0$.

Proof

It is Theorem 12 in Chap. 9 in Petrov (1975). $\square$

Lemma 2

Suppose that $\{\xi _i\}$ is a sequence of i.i.d. random variables with $\mathrm{E}|\xi _1|<\infty$, $\{a_i\}_{i=1}^k$ is a sequence of positive real numbers. Denote $A_k=\sum _{i=1}^k a_i$ and $B_k=\#\{i:A_i/a_i \le k\}/k$ with $\#\{i:A_i/a_i \le k\}$ being the number of i satisfying $A_i/a_i \le k$, and then $\sum _{i=1}^k a_i \xi _i/A_k \mathop {\rightarrow }\limits ^{a.s.} \mathrm{E} \xi _1$ if $A_k \rightarrow \infty$; in particular, if $c_1\le a_i \le c_2$ for some positive constants $c_1,c_2$, $\sum _{i=1}^k a_i \xi _i/A_k \mathop {\rightarrow }\limits ^{a.s.} \mathrm{E}\xi _1$.

Proof

It is the theorem in Jamison et al. (1965). $\square$

Lemma 3

(1) Suppose conditions C.1, C.2, C.3, C.4(1) hold. Then $n^{-1} {\mathbf{g}}^{(1)}({\varvec{\beta }})+{\varvec{\Sigma }}_{S_X}({\varvec{\beta }}) \mathop {\rightarrow }\limits ^{a.s.} 0$ under $H_{0G}$; $n^{-1} \tilde{\mathbf{g}}_{(1)}^{(1)}({\varvec{\beta }})+{\varvec{\Sigma }}_{\tilde{S}_X(1)}({\varvec{\beta }})\mathop {\rightarrow }\limits ^{a.s.} 0$ under $H_{0}$, and $n^{-1} \tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})+{\varvec{\Sigma }}_{\tilde{S}_X}({\varvec{\beta }})\mathop {\rightarrow }\limits ^{a.s.} 0$ under $H_{A}$. All converge uniformly in ${\varvec{\beta }} \in {\mathscr {B}}$;

(2) Under conditions C.1, C.2, C.4(1),

$$\begin{aligned} \tilde{\mathbf{f}}({\mathbf{X}},\hat{{\varvec{\beta }}}_f)-\tilde{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})= & {} -\tilde{\mathbf{S}}_X({\varvec{\beta }}) \left\{ \tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})\right\} ^{-1} \{\tilde{\mathbf{S}}_X({\varvec{\beta }})\}^\top \mathbf{e} +o(\hat{{\varvec{\beta }}}_f-{\varvec{\beta }}) a.s. \\ \tilde{\mathbf{f}}_{(1)}({\mathbf{X}},\hat{{\varvec{\beta }}}_0)-\tilde{\mathbf{f}}_{(1)}({\mathbf{X}},{\varvec{\beta }})= & {} -\tilde{\mathbf{S}}_{(1)}({\mathbf{X}},{\varvec{\beta }}) \left\{ \tilde{\mathbf{g}}_{(1)}^{(1)}({\varvec{\beta }})\right\} ^{-1}\!\! \{\tilde{\mathbf{S}}_{(1)}({\mathbf{X}},{\varvec{\beta }})\}^\top {\mathbf{e}} +o(\hat{{\varvec{\beta }}}_0-{\varvec{\beta }}) a.s. \\ {\mathbf{f}}({\mathbf{X}},\hat{{\varvec{\beta }}}_{0G})-{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})= & {} -{\mathbf{S}}_X({\varvec{\beta }}) \left\{ {\mathbf{g}}^{(1)}({\varvec{\beta }})\right\} ^{-1} \{{\mathbf{S}}_X({\varvec{\beta }})\}^\top {\mathbf{e}} +o(\hat{{\varvec{\beta }}}_{0G}-{\varvec{\beta }}) a.s. \end{aligned}$$

Proof

(1) Recall the notation of ${\mathbf{g}}^{(1)}({\varvec{\beta }})$, denote ${\mathbf{II}}_1^{g(1)} ={\mathbf{I}}_p\otimes \{{\mathbf{Y}}-{\mathbf{f}}({\mathbf{X}},{\varvec{\beta }})\}^\top {\mathbf{S}}_X^{(1)}({\varvec{\beta }})$, let ${\mathbf{II}}_{1k}^{g(1)}$ be the kth row of ${\mathbf{II}}_1^{g(1)}$ for $k=1,2,\ldots ,p$, and then it is sufficient to verify $n^{-1} {\mathbf{II}}_{1k}^{g(1)} \mathop {\rightarrow }\limits ^{a.s.} 0$ under $H_{0G}$. In fact, conditions C.3, C.4(1) indicate that $n^{-1}{\mathbf{II}}_{1k}^{g(1)}= n^{-1} \sum _{i=1}^m \sum _{j=1}^{n_i} {\mathbf{S}}_{x_{ij}}({\varvec{\beta }}) \varepsilon _{ij} \mathop {\rightarrow }\limits ^{a.s.} 0$. Similarly, the conclusions about $n^{-1} \tilde{\mathbf{g}}_{(1)}^{(1)}({\varvec{\beta }})$ under $H_0$ and $n^{-1} \tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})$ under $H_A$ in Lemma 3 (1) can be derived, respectively.

(2) According to Theorem 1 (1), under conditions C.1, C.2, C.4(1), ${\tilde{{\varvec{\beta }}}}\mathop {\rightarrow }\limits ^{a.s.} \beta$ where ${\tilde{{\varvec{\beta }}}}$ is the estimate of ${{\varvec{\beta }}}$, which may be ${\hat{{\varvec{\beta }}}}_f$, ${\hat{{\varvec{\beta }}}}_0$ or $\hat{{\varvec{\beta }}}_{0G}$ defined in (8). Then by the Taylor expansions, $\mathbf{f}(\mathbf{X},{\tilde{{\varvec{\beta }}}})=\mathbf{f}(\mathbf{X},{{\varvec{\beta }}})+\mathbf{S}_X({{\varvec{\beta }}}) ({\tilde{{\varvec{\beta }}}}-{\varvec{\beta }})+o({\tilde{{\varvec{\beta }}}}-{\varvec{\beta }})$ a.s., and

$$\begin{aligned} 0=\tilde{\mathbf{g}}(\hat{{\varvec{\beta }}}_f)= & {} \tilde{\mathbf{g}}({\varvec{\beta }})+\tilde{\mathbf{g}}^{(1)}({\varvec{\beta }})(\hat{{\varvec{\beta }}}_f-{\varvec{\beta }})+o(\hat{{\varvec{\beta }}}_f-{\varvec{\beta }}) \quad a.s., \\ 0=\tilde{\mathbf{g}}_{(1)}(\hat{{\varvec{\beta }}}_0)= & {} \tilde{\mathbf{g}}_{(1)}({\varvec{\beta }})+\tilde{\mathbf{g}}_{(1)}^{(1)}({\varvec{\beta }})(\hat{{\varvec{\beta }}}_0-{\varvec{\beta }})+o(\hat{{\varvec{\beta }}}_f-{\varvec{\beta }}) \quad a.s.,\\ 0={\mathbf{g}}(\hat{{\varvec{\beta }}}_{0G})= & {} {\mathbf{g}}({\varvec{\beta }})+{\mathbf{g}}^{(1)}({\varvec{\beta }})(\hat{{\varvec{\beta }}}_{0G}-{\varvec{\beta }})+o(\hat{{\varvec{\beta }}}_{0G}-{\varvec{\beta }}) \quad a.s., \end{aligned}$$

which yield the conclusions of Lemma 3(2) by recalling the notations of $\tilde{\mathbf{g}}({\varvec{\beta }})$, $\tilde{\mathbf{g}}_{(1)}({\varvec{\beta }})$ and ${\mathbf{g}}({\varvec{\beta }})$, respectively. $\square$

Lemma 4

For the error ${\varepsilon }$ in model (1) in the text, which has independent components, we have the following conclusions for any $n\times n$ symmetric matrix $\mathbf A _n$,

• (i) $\text{ var }({{\varvec{\varepsilon }}}^\top \mathbf A _n {{\varvec{\varepsilon }}})= (\kappa -3 \sigma ^4 ) \cdot \text{ tr }\{\text{ diag }^2(\mathbf A _n )\}+ 2 \sigma ^4 \cdot \text{ tr }(\mathbf A _n^2 )$;

• (ii) ${\nu _{max}(\mathbf{A}_n^2)}/{\mathrm{tr}(\mathbf{A}_n^2)}\rightarrow 0$, with $\nu _{max}(\mathbf{A}_n^2)$ being the maximal eigenvalue of the matrix $\mathbf{A}_n^2$, implies

  $$\begin{aligned} \frac{{{\varvec{\varepsilon }}}^\top \mathbf A _n {{\varvec{\varepsilon }}}- \sigma ^2 \cdot \text{ tr } (\mathbf A _n)}{\sqrt{ (\kappa -3 \sigma ^4 ) \cdot \mathrm{tr}\{\text{ diag }^2(\mathbf A _n)\}+2 \sigma ^4 \cdot \mathrm{tr}(\mathbf A _n^2)}} \mathop {\rightarrow }\limits ^{d} \text{ N }(0, 1). \end{aligned}$$

Proof

This lemma is Lemma 3 in the supplement of Li et al. (2014). $\square$

Lemma 5

Assume that ${\varvec{\varsigma }}=({\varsigma }_1,{\varsigma }_2,\ldots ,{\varsigma }_k)^\top$ is a $k\times 1$ random vector with its elements having the fourth-order moments, and then for any $k\times k$ matrix ${ \mathbf M}$, $\mathrm{E}({\varvec{\varsigma }} ^\top \mathbf{M}{\varvec{\varsigma }})^2= \mathrm{vec}^\top (\mathbf{M}^\top \otimes \mathbf{M}) \cdot {\varvec{\gamma }}_{4\zeta }$ with $\gamma _{4\zeta }\equiv \mathrm{E}\{({\varvec{\varsigma }} \otimes {\varvec{\varsigma }} )^{\otimes 2}\}$.

Proof

Note that

$$\begin{aligned} \mathrm{E}({\varvec{\varsigma }} ^\top \mathbf M {\varvec{\varsigma }})^2= & {} \mathrm{E}\left[ \text{ vec }\{{\varvec{\varsigma }} ^\top ({\mathbf{M}} {\varvec{\varsigma }} {\varvec{\varsigma }} ^\top \mathbf M ){\varvec{\varsigma }} \}\right] = \mathrm{E} \{({\varvec{\varsigma }} ^\top \otimes {\varvec{\varsigma }} ^\top ) \text{ vec }(\mathbf M {\varvec{\varsigma }} {\varvec{\varsigma }} ^\top \mathbf M )\}\\= & {} \mathrm{E}\left[ ({\varvec{\varsigma }} ^\top \otimes {\varvec{\varsigma }} ^\top ) (\mathbf M ^\top \otimes \mathbf M ) \text{ vec }({\varvec{\varsigma }} {\varvec{\varsigma }} ^\top )\right] \\= & {} \mathrm{E}\left[ \text{ vec }\{ ({\varvec{\varsigma }} ^\top \otimes {\varvec{\varsigma }} ^\top ) (\mathbf M ^\top \otimes \mathbf M ) \text{ vec }({\varvec{\varsigma }} {\varvec{\varsigma }} ^\top ) \}\right] \\= & {} \left[ \mathrm{E}\{\text{ vec }({\varvec{\varsigma }} {\varvec{\varsigma }} ^\top )\}^\top \otimes ({\varvec{\varsigma }} ^\top \otimes {\varvec{\varsigma }} ^\top ) \text{ vec }(\mathbf M ^\top \otimes \mathbf M ) \right] \\= & {} \mathrm{E}\{({\varvec{\varsigma }}^\top \otimes {\varvec{\varsigma }}^\top )^{\otimes 2}\} \cdot \text{ vec }(\mathbf M ^\top \otimes \mathbf M ) \\= & {} \text{ vec }^\top (\mathbf M ^\top \otimes \mathbf M ) \cdot {\varvec{\gamma }}_{4\zeta }. \end{aligned}$$

$\square$

Lemma 6

Suppose ${\varvec{\xi }}=({\varvec{\xi }}_1^\top ,\ldots ,{\varvec{\xi }}_m^\top )^\top$ is a $q \times 1$ random vector where ${\varvec{\xi }}_i's$ are independent $q_i\times 1$ sub-vector for different i with mean 0 and covariance matrix ${\varvec{\Sigma }}_i$. For any $q\times q$ symmetric diagonal block matrix $\mathbf{A}_q=\mathrm{diag}({\mathbf{A}}_{11},\ldots ,{\mathbf{A}}_{mm})$ with the ith block ${\mathbf{A}}_{ii}$ being $q_i\times q_i$,

• (i) $\text{ var }({\varvec{\xi }}^\top \mathbf{A }_q {{\varvec{\xi }}})= \sum _{i=1}^m \left[ \text{ vec }^\top ({\mathbf{A}}_{ii}^\top \otimes {\mathbf{A}}_{ii}) \cdot \mathrm{E}\{({\varvec{\xi }}_i \otimes {\varvec{\xi }}_i)^{\otimes 2}\}- \left\{ \text{ tr }({\mathbf{A}}_{ii} {\varvec{\Sigma }}_i)\right\} ^2 \right]$;

• (ii) Assume for different i, $q_i \equiv q_1$ and $\mathrm{E}\{({\varvec{\xi }}_i \otimes {\varvec{\xi }}_i)^{\otimes 2}\}\equiv {\varvec{\gamma }}_{4\xi _1}$. If all the elements of ${\varvec{\xi }}_i$ have the $(4+\delta )$th-order moments for some $\delta >0$, then

  $$\begin{aligned} \frac{{\varvec{\xi }}^\top \mathbf{A }_q {\varvec{\xi }}- \sum _{i=1}^m \text{ tr } ({\mathbf{A}}_{ii} {\varvec{\Sigma }}_i )}{\sqrt{ \text{ vec }^\top \left\{ \sum _{i=1}^m \left( {\mathbf{A}}_{ii}^\top \otimes {\mathbf{A}}_{ii}\right) \right\} \cdot {\varvec{\gamma }}_{4\xi _1}-\sum _{i=1}^m\left\{ \text{ tr }({\mathbf{A}}_{ii}{\varvec{\Sigma }}_i)\right\} ^2}} \mathop {\rightarrow }\limits ^{d} \text{ N }(0, 1). \end{aligned}$$

Proof

(i) Note that by Lemma 5

$$\begin{aligned} \text{ var }({\varvec{\xi }}^\top \mathbf{A }_q {\varvec{\xi }})= & {} \sum _{i=1}^m \text{ var }({\varvec{\xi }}_i^\top {\mathbf{A}}_{ii} {\varvec{\xi }}_i) \\= & {} \sum _{i=1}^m \left[ \mathrm{E}\{({\varvec{\xi }}_i^\top \otimes {\varvec{\xi }}_i^\top )^{\otimes 2}\} \text{ vec }({\mathbf{A}}_{ii}^\top \otimes {\mathbf{A}}_{ii})- \left\{ \text{ tr }({\mathbf{A}}_{ii} {\varvec{\Sigma }}_i)\right\} ^2 \right] . \end{aligned}$$

(ii) Let $\eta _i={\varvec{\xi }}_i^\top {\mathbf{A}}_{ii} {\varvec{\xi }}_i$, and then $\eta _i's$ are independent. Denote $B_m^2=\sum _{i=1}^m \text{ var }(\eta _i)$ which can be written as $B_m^2=\text{ vec }^\top \left\{ \sum _{i=1}^m \left( {\mathbf{A}}_{ii}^\top \otimes {\mathbf{A}}_{ii}\right) \right\} \cdot {\varvec{\gamma }}_{4\xi _1}-\sum _{i=1}^m\left\{ \text{ tr }({\mathbf{A}}_{ii}{\varvec{\Sigma }}_i)\right\} ^2$. According to the conditions in the lemma and Lyapunov cental limit theory (CLT), the conclusion is verified. $\square$

Proof of Theorem 1

(Asymptotic properties of estimates for ${\varvec{\beta }}$ and $\sigma ^2$):

(1) Note that ${\mathbf{P}}_{Z^{(1)^\bot }}$ and ${\mathbf{P}}_{Z^\bot }$ are idempotent matrices and the functions $\tilde{\mathbf{f}}_{(1)}(\cdot )$ and $\tilde{\mathbf{f}}(\cdot )$ satisfy conditions C.1, C.2(2), C.4(1), and then the strong consistency of $\hat{{\varvec{\beta }}}_0$ and $\hat{{\varvec{\beta }}}_f$ is obtained by Jennrich (1969).

(2) According to conditions C.1–C.3, C.4(1) and the proof of the asymptotic normality of $\hat{{\varvec{\beta }}}_{0G}$ in Jennrich (1969), the conclusions in Theorem 1(2) can be derived by the mean-value theorem and the convergence of the estimators $\hat{{\varvec{\beta }}}_f$ and $\hat{{\varvec{\beta }}}_0$ in Theorem 1(1). $\square$

Proof of Theorems 2–5 and Corollary 1:

See the Supplement. $\square$

Appendix 4: derivation of ${\varvec{\gamma }}_{4b}$

See the Supplement. $\square$