Consistent EM algorithm for a spatial autoregressive probit model

Abstract

This paper is concerned with the estimation of spatial autoregressive probit models, which are increasingly used in many empirical settings. Among existing estimators, the EM algorithm for spatial probit models introduced by McMillen (J Reg Sci 32(3):335–348, 1992) is a widely used method, but it lacks proof of consistency. In this paper, we formally show that it is inconsistent by applying the law of large numbers for dependent and non-identically distributed near-epoch dependence (NED) random fields. We provide a modification of the EM algorithm to yield a consistent estimator. Monte Carlo experiments show that in finite samples, our new EM algorithm outperforms McMillen’s EM algorithm, especially for medium to high levels of spatial dependence.

Appendix

1.1 Notations

• n : sample size

• $\{\varepsilon _{i,n}\}_{i=1}^n$ : i.i.d. normal error term

• $\{y_{i,n}\}_{i=1}^n$ : observed binary outcome variable

• $\{y^*_{i,n}\}_{i=1}^n$ : continuous latent variable

• $U_n= (I_{n}-\rho W_{n})^{-1}\varepsilon _{n}$

• $\theta =(\beta ',\rho )'$ : parameters

• $x_{i,n}^{*'}$ : the $i^{th}$ row of $X_{n}^*\equiv (I_{n}-\rho W_{n})^{-1}X_{n}$

• $z_{i,n}(\theta ^{+})=E(y_{i,n}^{*}|y_{i,n},\theta ^{+})$

• $\Upsilon _{n}(\rho )=Var(Y_{n}^{*})$

• $\sigma _{ii,n}=\sqrt{\Upsilon _{ii,n}(\rho )}$, where $\Upsilon _{ii,n}(\rho )$ is the element on the $i^{th}$ row and $i^{th}$ column of $\Upsilon _{n}(\rho )$

• $\Gamma (\theta ^{+},\theta _0)=(\omega _{1,n}(\theta ^{+},\theta _0),\ldots ,\omega _{n,n}(\theta ^{+},\theta _0))'$

• $A_{1}=\lim _{n\rightarrow \infty } \frac{1}{n}X_{n}'X_{n}$

• $\Lambda (\theta ^{+},\theta _0)=Var(Z_{n}(\theta ^{+})|\theta _0)$

1.2 Proofs

Lemma

Under Assumption 1,

• (i) $\{u_{i,n}\}_{i=1}^{n}$is uniformly $L_{p}$ bounded, i.e $sup_{i,n}E|u_{i,n}|^{p}<\infty$ where p is a positive integer.

• (ii) $\{u_{i,n}\}_{i=1}^{n}$ is a uniformly and geometrically $L_{2}$-NED random field on $\{\varepsilon _{i,n}\}_{i=1}^{n}$ with NED coefficient $\zeta ^{s/\bar{d}}$.

Proof

If $\{\varepsilon _{i,n}\}_{i=1}^{n}$ are independent and identically distributed with $E|\varepsilon _{i,n}|^{p}<\infty$, $M=(m_{i,j,n})$ is a nonstochastic $n\times n$ matrix with $||M||_{\infty }<B,$ then $E[(\sum _{j=1}^{n}|m_{i,j,n}\varepsilon _{j,n}|)^{p}]\le E|\varepsilon _{1,n}|^{p}B^{p}$ (Xu and Lee 2015).

Recall that $U_{n}=(I_{n}-\rho W_{n})^{-1}\varepsilon _{n}$. Let $M_{n}=(I_{n}-\rho W_{n})^{-1}$, we have $u_{i,n}=\sum _{j=1}^{n}m_{i,j,n}\varepsilon _{j,n}$.

$$\begin{aligned} ||M_{n}||_{\infty }= & {} ||\sum _{l=0}^{n}\rho ^{l}W_{n}^{l}||_{\infty }\\\le & {} \sum _{l=0}^{n}||\rho ^{l}W_{n}^{l}||_{\infty }\\\le & {} \sum _{l=0}^{n}\zeta ^{l}=\frac{1}{1-\zeta }. \end{aligned}$$

In addition, $\{\varepsilon _{i,n}\}_{i=1}^{n}$ is i.i.d normal distributed. Hence $E|u_{i,n}|^{p}\le E|\varepsilon _{1,n}|^{p}(\frac{1}{1-\zeta })^{p}$, and $sup_{i,n}E|u_{i,n}|^{p}\le sup_{n}E|\varepsilon _{1,n}|^{p}(\frac{1}{1-\zeta })^{p}<\infty$.

Next, we construct a sequence: $U_{n}^{(0)}=\iota _{n}\equiv (1,1,\ldots ,1)'$, $U_{n}^{(m+1)}=\rho W_{n}U_{n}^{(m)}+\varepsilon _{n}$, $\forall m=0,1,2,\ldots ,\infty$. Since $U_{n}=(I_{n}-\rho W_{n})^{-1}\varepsilon _{n}=\sum _{j=1}^{\infty }\rho ^{j}W_{n}^{j}\varepsilon _{n}$, it follows that $U_{n}^{(m)}\rightarrow U_{n}$ as $m\rightarrow \infty$.

$$\begin{aligned} |u_{i,n}^{(m+1)}-u_{i,n}^{(m)}|&=|\rho \sum _{j=1}^{n}w_{i,j,n}(u_{j,n}^{(m)}-u_{j,n}^{(m-1)})|\le \rho \sum _{j=1}^{n}w_{i,j,n}|u_{j,n}^{(m)}-u_{j,n}^{(m-1)}|.\\ |u_{i,n}^{(m+1)}-u_{i,n}^{(m)}|^{2}&\le \rho ^{2}\sum _{j=1}^{n}\sum _{k=1}^{n}w_{i,j,n}w_{i,k,n}|u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||u_{k,n}^{(m)}-u_{k,n}^{(m-1)}|.\\ E|u_{i,n}^{(m+1)}-u_{i,n}^{(m)}|^{2}&\le \rho ^{2}\sum _{j=1}^{n}\sum _{k=1}^{n}w_{i,j,n}w_{i,k,n}E[|u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||u_{k,n}^{(m)}-u_{k,n}^{(m-1)}|]\\&\le \rho ^{2}\sum _{j=1}^{n}\sum _{k=1}^{n}w_{i,j,n}w_{i,k,n}||u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||_{2}\cdot ||u_{k,n}^{(m)}-u_{k,n}^{(m-1)}||_{2}\\&\le \rho ^{2}||W_{n}||_{\infty }^{2}\{max_{1\le j\le n}\{||u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||_{2}\}\}^{2}, \end{aligned}$$

where the penultimate inequality is by Holder’s inequality: $E|XY|\le ||X||_{2}||Y||_{2}$.

$$\begin{aligned} ||u_{i,n}^{(m+1)}-u_{i,n}^{(m)}||_{2}\le \rho ||W_{n}||_{\infty }max_{1\le j\le n}\{||u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||_{2}\}\le \zeta \cdot max_{1\le j\le n}\{||u_{j,n}^{(m)}-u_{j,n}^{(m-1)}||_{2}\}. \end{aligned}$$

Since only individuals located within a distance of $\bar{d}$ have influence on each other, $u_{i,n}^{(m)}$ is a function of $\varepsilon _{j,n}$’s where $d(i,j)\le m\bar{d}$. Therefore,

$$\begin{aligned} ||u_{i,n}-E(u_{i,n}|\mathcal {F}_{i,n}(m\bar{d})||_{2}\le ||u_{i,n}-u_{i,n}^{(m)}||_{2}\le \zeta \cdot max_{1\le j\le n}\{||u_{j,n}-u_{j,n}^{(m-1)}||_{2}\}\le \zeta ^{m}c_{n}, \end{aligned}$$

where $c_{n}=max_{1\le j\le n}\{||u_{j,n}-u_{j,n}^{(0)}||_{2}\}\le sup_{1\le j\le n}||u_{j,n}||_{2}+1$. By Lemma 1, $c_{n}<\infty$. Hence, $||u_{i,n}-E(u_{i,n}|\mathcal {F}_{i,n}(s)||_{2}\le \zeta ^{s/\bar{d}}c_{n}$, where $\zeta \in (0,1)$ and $\bar{d}>1$. Therefore, $\{u_{i,n}\}_{i=1}^{n}$ is a uniformly and geometrically $L_{2}$-NED random field on $\{\varepsilon _{i,n}\}_{i=1}^{n}$. $\square$

Lemma

Under Assumption 1, $\{y_{i,n}^{*}\}_{i=1}^{n}$, $\{y_{i,n}\}_{i=1}^{n}$, $\{z_{i,n}(\theta ^{+})\}_{i=1}^{n}$, $\{x_{i,n}'\beta \cdot W_{i,n}Z_{n}(\theta ^{+})\}_{i=1}^{n}$, $\{z_{i,n}(\theta ^{+})^{2}\}_{i=1}^{n}$, $\{z_{i,n}(\theta ^{+})W_{i,n}Z_{n}(\theta ^{+})\}_{i=1}^{n}$, $\{z_{i,n}(\theta ^{+})W_{i,n}^{c}Z_{n}(\theta ^{+})\}_{i=1}^{n}$, $\{z_{i,n}(\theta ^{+})W_{i,n}^{s}Z_{n}(\theta ^{+})\}_{i=1}^{n}$ are uniformly and geometrically $L_{2}$-NED random fields on $\{\varepsilon _{i,n}\}_{i=1}^{n}$.

Proof

Given that $y_{i,n}^{*}=x_{i,n}^{*'}\beta +u_{i,n}$, $x_{i,n}^{*'}\beta$ is non-stochastic and uniformly bounded, and $\{u_{i,n}\}_{i=1}^{n}$ is shown to be an NED random field, it follows that $\{y_{i,n}^{*}\}_{i=1}^{n}$ is an NED random field. Hence, as in Proposition 2 of Xu and Lee (2015), $y_{i,n}=\varvec{1}(y_{i,n}^{*}>0)$ is an NED random field.

For brevity, we suppress z’s dependence on $\theta ^{+}$ in the notations.

$$\begin{aligned} z_{i,n}&=E(y_{i,n}^{*}|y_{i,n})\\&=x_{i,n}^{*'}\beta +\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}y_{i,n}-\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{1-\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}(1-y_{i,n})\\&=x_{i,n}^{*'}\beta +\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})[1-\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})]}y_{i,n}-\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{1-\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}. \end{aligned}$$

Given that $\{y_{i,n}\}_{i=1}^{n}$ is an NED random field, and that $z_{i,n}$ is a linear transformation of $y_{i,n}$, to prove that $\{z_{i,n}\}_{i=1}^{n}$ is also an NED random field requires that i) $\sigma _{ii,n}$ is uniformly bounded, ii) $\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})$ is uniformly bounded and iii) $\exists c_1,c_2$ such that $0<c_1<\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})<c_2<1$. Considering Assumption 1(4)–(5), and

$$\begin{aligned} \sigma _{ii,n}&=\sqrt{[(I_{n}-\rho W_{n})^{-1}(I_{n}-\rho W_{n})^{-1'}]_{i,i}}\\&\le ||(I_{n}-\rho W_{n})^{-1}(I_{n}-\rho W_{n})^{-1'}||_{\infty }\\&\le ||(I_{n}-\rho W_{n})^{-1}||_{\infty }||(I_{n}-\rho W_{n})^{-1'}||_{\infty }\\&\le \frac{1}{(1-\zeta )(1-\varsigma )}\\&<+\infty . \end{aligned}$$

It follows that $\{z_{i,n}\}_{i=1}^{n}$ is also an NED random field.

Since $\{z_{i,n}\}_{i=1}^{n}$ is a uniformly and geometrically $L_{2}-$NED random field, by definition, there exists an array of finite postie constants $\{d_{i,n},i\in D_{n},i\ge 1\}$ where $\sup _{n}\sup _{i\in D_{n}}d_{i,n}<\infty$ and a sequence $\psi (s)=O(p^{s})$ where $0<p<1$ such that $||z_{i,n}-E(z_{i,n}|\mathcal {F}_{i,n}(s))||_{2}\le d_{i,n}\psi (s)$.

$$\begin{aligned} ||x_{i,n}'\beta \cdot W_{i,n}Z_{n}-x_{i,n}'\beta \cdot E(W_{i,n}Z_{n}|\mathcal {F}_{i,n}(s))||_{2}&\le |x_{i,n}'\beta |\cdot \sum _{j=1}^{n}w_{i,j,n}||z_{i,n}-E(z_{i,n}|\mathcal {F}_{i,n}(s))||_{2}\\&\le |x_{i,n}'\beta |\cdot \sum _{j=1}^{n}w_{i,j,n}d_{i,n}\psi (s)\\&\le d_{max}\psi (s)||W_{n}||_{\infty }, \end{aligned}$$

where $d_{max}=\sup _{i,n}\{|x_{i,n}'\beta |\cdot d_{i,n}\}<\infty$, and $||W_{n}||<\infty$, so $\{W_{i,n}Z_{n}\}_{i=1}^{n}$ is also a uniformly and geometrically $L_{2}$-NED random field on $\{\varepsilon _{i,n}\}_{i=1}^{n}$.

By Lemma A.4 in Xu and Lee (2015), “$G(x):Domain\rightarrow R$ satisfies $|G(x_{1})-G(x_{2})|\le C_{1}(|x_{1}|^{a}+|x_{2}|^{a}+1)|x_{1}-x_{2}|$ for some integer $a\ge 1$. If $\{\nu _{i,n}\}_{i=1}^{n}$ is a random field with $||\nu _{i,n}-E(\nu _{i,n}|\mathcal {F}_{i,n}(s))||_{2}\le C_{2}\psi (s)$ for all i and n, and $sup_{i,n}||\nu _{i,n}||_{p}<\infty$ for some $p>2a+2$. Then $||G(\nu _{i,n})-E(G(\nu _{i,n})|\mathcal {F}_{i,n}(s))||_{2}\le C\psi (s)^{(p-2a-2)/(2p-2a-2)}$.”

In this case, $G(z)=z^{2}$. $|G(z_{1})-G(z_{2})|=|z_{1}^{2}-z_{2}^{2}|=|z_{1}+z_{2}|\cdot |z_{1}-z_{2}|\le (|z_{1}|+|z_{2}|+1)\cdot |z_{1}-z_{2}|$. And we know $sup_{i,n}||z_{i,n}||_{p}<\infty$ for any finite integer p, so the above lemma holds with $a=1$. $||z_{i,n}^{2}-E(z_{i,n}^{2}|\mathcal {F}_{i,n}(s))||_{2}\le C\psi (s)^{\frac{p-4}{2p-4}}$.

Since $\{z_{i,n}\}_{i=1}^{n}$ is an NED random field, by definition, there exist constants $R_{1}<+\infty$ and $h\in (0,1)$ such that $||z_{i,n}-E(z_{i,n}|\mathcal {F}_{i,n}(s))||_{2}\le R_{1}h^{s}$. Because spatial correlation is zero for two individuals whose distance is greater than $\bar{d}$, $z_{i,n}W_{i,n}Z_{n}=z_{i,n}\sum _{j=1}^{n}w_{i,j,n}z_{j,n}=\sum _{\{j:d(i,j)<\bar{d}\}}w_{i,j,n}z_{i,n}z_{j,n}$. For any j such that $d(i,j)<\bar{d}$, and any $m>1$,

$$\begin{aligned}&||z_{i,n}z_{j,n}-E(z_{i,n}z_{j,n}|\mathcal {F}_{i,n}(m\bar{d})||_{2}\\&\le ||z_{i,n}z_{j,n}-E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))E(z_{j,n}|\mathcal {F}_{i,n}(m\bar{d}))||_{2}\\&=||z_{i,n}z_{j,n}-z_{j,n}E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))+z_{j,n}E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))-E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))E(z_{j,n}|\mathcal {F}_{i,n}(m\bar{d}))||_{2}\\&\le |z_{j,n}|\cdot ||z_{i,n}-E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))||_{2}+|E(z_{i,n}|\mathcal {F}_{i,n}(m\bar{d}))|\cdot ||z_{j,n}-E(z_{j,n}|\mathcal {F}_{i,n}(m\bar{d}))||_{2}\\&\le RR_{2}h^{m\bar{d}}+R_{3}||z_{j,n}-E(z_{j,n}|\mathcal {F}_{j,n}((m-1)\bar{d}))||_{2}\\&\le RR_{2}h^{m\bar{d}}+RR_{3}h^{(m-1)\bar{d}}\\&=R_{4}h^{m\bar{d}}, \end{aligned}$$

where $R_{2}=\sup _{i,n}|z_{i,n}|<+\infty$, $R_{3}=\sup _{i,n,s}|E(z_{i,n}|\mathcal {F}_{i,n}(s))|<+\infty$, $R_{4}=RR_{2}+RR_{3}\frac{1}{\bar{d}}<+\infty$. Thus, $||z_{i,n}z_{j,n}-E(z_{i,n}z_{j,n}|\mathcal {F}_{i,n}(s)||_{2}\le R_{4}h^{s}$. It follows that

$$\begin{aligned} ||z_{i,n}W_{i,n}Z_{n}-E(z_{i,n}W_{i,n}Z_{n}|\mathcal {F}_{i,n}(s))||_{2}&\le \sum _{\{j:d(i,j)<\bar{d}\}}w_{i,j,n}||z_{i,n}z_{j,n}-E(z_{i,n}z_{j,n}|\mathcal {F}_{i,n}(s))||_{2}\\&\le R_{4}||W_{n}||_{\infty }h^{s}\\&\le \frac{R_{4}}{\rho _{m}}h^{s}. \end{aligned}$$

Therefore, $\{z_{i,n}W_{i,n}Z_{n}\}_{i=1}^{n}$ is a uniformly and geometrically $L_{2}$-NED random field. The same proof applies to $\{z_{i,n}W_{i,n}^{c}Z_{n}\}_{i=1}^{n}$ and $\{z_{i,n}W_{i,n}^{s}Z_{n}\}_{i=1}^{n}$. $\square$

Proposition

Under Assumption 1, assuming $\frac{1}{n}X_{n}'X_{n}\overset{}{\rightarrow }A_{1}$ then $\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})\overset{p}{\rightarrow }\tilde{Q}_{\infty }(\theta |\theta ^{+})$ pointwisely in $(\theta ,\theta ^{+})\in \Theta \times \Theta$, where $\tilde{Q}_{\infty }(\theta |\theta ^{+})= -\frac{1}{2}(\ln 2\pi )+\lim _{n\rightarrow \infty }\frac{1}{n}\ln |I_{n}-\rho W_{n}| -\frac{1}{2}(\beta ^{+}-\beta )'A_{1}(\beta ^{+}-\beta ) -\lim _{n\rightarrow \infty }\frac{1}{2n} \Gamma (\theta ^{+},\theta _0)'(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Gamma (\theta ^{+},\theta _0) -\lim _{n\rightarrow \infty }\frac{1}{n} \Gamma (\theta ^{+},\theta _0)'(I_{n}-\rho W_{n})'X_n(\beta ^{+}-\beta ) -\lim _{n\rightarrow \infty }\frac{1}{2n}tr[(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Lambda (\theta ^{+},\theta _0)]$, and $\Lambda (\theta ^{+},\theta _0)=Var(Z_{n}(\theta ^{+})|\theta _0)$.

Proof

Reorganizing the terms in Eq. (12), we have

$$\begin{aligned} \tilde{Q}_{n}(\theta |\theta ^{+})=-\frac{n}{2}(\ln 2\pi )+\ln |I_{n}-\rho W_{n}|-\frac{1}{2}((I_{n}-\rho W_{n})Z_{n}(\theta ^{+})-X_{n}\beta )'\cdot ((I_{n}-\rho W_{n})Z_{n}(\theta ^{+})-X_{n}\beta ). \end{aligned}$$

First,

$$\begin{aligned} \frac{1}{n}\beta 'X_{n}X_{n}\beta \overset{p}{\rightarrow }\beta 'A_{1}\beta . \end{aligned}$$

Second, by applying Theorem 1, we can obtain

$$\begin{aligned}&\frac{1}{n}\beta 'X_{n}'[Z_{n}(\theta ^{+})-\rho W_{n}Z_{n}(\theta ^{+})]\\&=\frac{1}{n}\sum _{i=1}^{n}x_{i,n}'\beta [z_{i,n}(\theta ^{+})-\rho W_{i,n}Z_{n}(\theta ^{+})]\\&\overset{p}{\rightarrow }\lim _{n\rightarrow \infty }\frac{1}{n}\sum _{i=1}^{n}x_{i,n}'\beta \cdot E[z_{i,n}(\theta ^{+})-\rho W_{i,n}Z_{n}(\theta ^{+})]\\&\overset{p}{\rightarrow }\lim _{n\rightarrow \infty }\frac{1}{n}\beta 'X_{n}(I_{n}-\rho W_{n})(X_{n}^{*'}\beta ^{+}+\Gamma (\theta ^{+},\theta _0))\\&=\beta 'A_{1}\beta ^{+}+\lim _{n\rightarrow \infty }\frac{1}{n}\beta 'X_{n}'(I_{n}-\rho W_{n})\Gamma (\theta ^{+},\theta _0). \end{aligned}$$

Third,

$$\begin{aligned}&\frac{1}{n}[Z_{n}(\theta ^{+})'Z_{n}(\theta ^{+})-\rho Z_{n}(\theta ^{+})'W_{n}'Z_{n}(\theta ^{+})-\rho Z_{n}(\theta ^{+})'W_{n}Z_{n}(\theta ^{+})+\rho ^{2}Z_{n}(\theta ^{+})'W_{n}'W_{n}Z_{n}(\theta ^{+})]\\&=\frac{1}{n}\sum _{i=1}^{n}[z_{i,n}^{2}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}Z_{n}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}^{c}Z_{n}(\theta ^{+})+\rho ^{2}z_{i,n}(\theta ^{+})W_{i,n}^{s}Z_{n}(\theta ^{+})]\\&\overset{p}{\rightarrow }\lim _{n\rightarrow \infty }\frac{1}{n}\sum _{i=1}^{n}E[z_{i,n}^{2}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}Z_{n}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}^{c}Z_{n}(\theta ^{+})+\rho ^{2}z_{i,n}(\theta ^{+})W_{i,n}^{s}Z_{n}(\theta ^{+})]. \end{aligned}$$

Since $Var(z_{i,n}(\theta ^{+}))=(\sigma _{ii,n}^{+})^{2}\frac{\phi (x_{i,n}^{*'}\beta ^{+}/\sigma _{ii,n}^{+})^{2}}{\Phi (x_{i,n}^{*'}\beta ^{+}/\sigma _{ii,n}^{+})^2[1-\Phi (x_{i,n}'\beta ^{+}/\sigma _{ii,n}^{+})]^2}\Phi (x_{i,n}^{*'}\beta _0/\sigma _{ii,0}^{+})[1-\Phi (x_{i,n}^{*'}\beta _0/\sigma _{ii,0})]$ and $E[z_{i,n}(\theta ^{+})^{2}]=(x_{i,n}^{*'}\beta ^{+})^{2}+2\omega _{i,n}(\theta ^{+})x_{i,n}^{*'}\beta ^{+}+\omega _{i,n}(\theta ^{+})^2+Var(z_{i,n}(\theta ^{+}))$, define $\Lambda (\theta ^{+},\theta _0)=Var(Z_{n}(\theta ^{+}))$, we have

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}E[z_{i,n}(\theta ^{+})^{2}] =&\frac{1}{n}\beta ^{+'}X_{n}^{*'}X_{n}^*\beta ^{+}+\frac{2}{n}\beta ^{+'}X_{n}^{*'}\Gamma (\theta ^{+},\theta _0)\\&+\frac{1}{n}\Gamma (\theta ^{+},\theta _0)'\Gamma (\theta ^{+},\theta _0)+\frac{1}{n}tr(\Lambda (\theta ^{+},\theta _0)). \end{aligned}$$

Henceforth,

$$\begin{aligned}&\lim _{n\rightarrow \infty }\frac{1}{n}\sum _{i=1}^{n}E[z_{i,n}^{2}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}Z_{n}(\theta ^{+})-\rho z_{i,n}(\theta ^{+})W_{i,n}^{c}Z_{n}(\theta ^{+})+\rho ^{2}z_{i,n}(\theta ^{+})W_{i,n}^{s}Z_{n}(\theta ^{+})]\\&=\beta ^{+'}A_{1}\beta ^{+}+\lim _{n\rightarrow \infty }\frac{2}{n}\beta ^{+'}X_{n}'(I_{n}-\rho W_{n})\Gamma (\theta ^{+},\theta _0)+\lim _{n\rightarrow \infty }\frac{1}{n}tr[(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Lambda (\theta ^{+},\theta _0)] \\&+\lim _{n\rightarrow \infty }\frac{1}{n}\Gamma (\theta ^{+},\theta _0)'(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Gamma (\theta ^{+},\theta _0). \end{aligned}$$

Therefore, summing up all the terms we can get

$$\begin{aligned} \frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})&\overset{p}{\rightarrow } -\frac{1}{2}(\ln 2\pi )+\lim _{n\rightarrow \infty }\frac{1}{n}\ln |I_{n}-\rho W_{n}| -\frac{1}{2}(\beta ^{+}-\beta )'A_{1}(\beta ^{+}-\beta ) \\&-\lim _{n\rightarrow \infty }\frac{1}{2n} \Gamma (\theta ^{+},\theta _0)'(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Gamma (\theta ^{+},\theta _0)\\&-\lim _{n\rightarrow \infty }\frac{1}{n} \Gamma (\theta ^{+},\theta _0)'(I_{n}-\rho W_{n})'X_n(\beta ^{+}-\beta ) \\&-\lim _{n\rightarrow \infty }\frac{1}{2n}tr[(I_{n}-\rho W_{n})'(I_{n}-\rho W_{n})\Lambda (\theta ^{+},\theta _0)]\\&=\tilde{Q}_{\infty }(\theta |\theta ^{+}). \end{aligned}$$

$\square$

Proposition

Under the assumptions in Proposition 1, $\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})\overset{p}{\rightarrow }\tilde{Q}_{\infty }(\theta |\theta ^{+})$ uniformly in $(\theta ,\theta ^{+})\in \Theta \times \Theta$, that is, $\sup _{(\theta ,\theta ^{+})\in \Theta \times \Theta }|\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})-\tilde{Q}_{\infty }(\theta |\theta ^{+})|=0$.

Proof

Because the parameter space is compact by assumption and pointwise convergence is given in Proposition 1, according to Theorem 1 in Donald (1992), we only need to show that $\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})$ is stochastically equicontinous in order to prove uniform convergence.

Examine each term in Eq. (12). First, using the mean value theorem, we have:

$$\begin{aligned} |\frac{1}{n}\ln |I_{n}-\rho _{1}W_{n}|-\frac{1}{n}\ln |I_{n}-\rho _{2}W_{n}||&=\frac{1}{n}|\frac{\partial \ln |I_{n}-\bar{\rho }W_{n}|}{\partial \rho }|\cdot |\rho _{1}-\rho _{2}|\\&=\frac{1}{n}|tr((I_{n}-\bar{\rho }W_{n})^{-1}W_{n})|\cdot |\rho _{1}-\rho _{2}|\\&\le \sup _{n}||(I_{n}-\bar{\rho }W_{n})^{-1}W_{n}||_{\infty }\cdot |\rho _{1}-\rho _{2}|\\&\le \sup _{n}||\sum _{j=0}^{n}\rho _{m}^{j}W_{n}^{j+1}||_{\infty }\cdot |\rho _{1}-\rho _{2}|\\&\le \frac{1}{\rho _{m}}\frac{\zeta }{1-\zeta }\cdot |\rho _{1}-\rho _{2}|, \end{aligned}$$

where $\bar{\rho }$ is between $\rho _{1}$ and $\rho _{2}$, and $\rho _{m}=\sup \{|\rho |:\rho \in \varvec{\rho }\}$.

$$\begin{aligned} |\frac{1}{n}\beta _{1}'X_{n}'X_{n}\beta _{1}-\frac{1}{n}\beta _{2}'X_{n}'X_{n}\beta _{2}|&\le \frac{2}{n}|(X_{n}'X_{n}\bar{\beta })'\cdot (\beta _{1}-\beta _{2})|\\&\le \frac{2}{n}||X_{n}'X_{n}\bar{\beta }||_{2}\cdot ||\beta _{1}-\beta _{2}||_{2}\\&=2||\frac{1}{n}\sum _{i=1}^{n}x_{i,n}x_{i,n}'\bar{\beta }||_{2}\cdot ||\beta _{1}-\beta _{2}||_{2}\\&\le 2M||\frac{1}{n}\sum _{i=1}^{n}x_{i,n}||_{2}\cdot ||\beta _{1}-\beta _{2}||_{2}, \end{aligned}$$

where the second inequality is from the Cauchy-Schwartz inequality; $M=\sup _{i,n,\beta }\{|x_{i,n}'\beta |\}<+\infty$. Since $X_{n}$ is uniformly bounded, so $2M||\frac{1}{n}\sum _{i=1}^{n}x_{i,n}||_{2}<+\infty$.

Let $\gamma =(\theta ',\theta ^{+'})'$,

$$\begin{aligned} |\frac{1}{n}\beta _{1}'X_{n}'Z_{n}(\theta _{1}^{+})-\frac{1}{n}\beta _{2}'X_{n}'Z_{n}(\theta _{2}^{+})|&\le \frac{1}{n}|\frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \gamma }|_{\gamma =\bar{\gamma }}\cdot |\gamma _{1}-\gamma _{2}|\\&\le \frac{1}{n}||\frac{\partial [\bar{\beta }'X_{n}'Z_{n}(\bar{\theta }^{+})]}{\partial \gamma }||_{2}\cdot ||\gamma _{1}-\gamma _{2}||_{2}, \end{aligned}$$

where $\bar{\gamma }$ is between $\gamma _{1}$ and $\gamma _{2}$.

$$\begin{aligned} \frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \beta }&=X_{n}'Z_{n}(\theta ^{+})=\sum _{i=1}^{n}x_{i,n}z_{i,n}(\theta ^{+})\\ \frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \rho }&=0\\ \frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \beta ^{+}}&=\sum _{i=1}^{n}x_{i,n}'\beta \cdot \frac{\partial z_{i,n}(\theta ^{+})}{\partial \beta ^{+}}\\ \frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \rho ^{+}}&=\sum _{i=1}^{n}x_{i,n}'\beta \cdot \frac{\partial z_{i,n}(\theta ^{+})}{\partial \rho ^{+}} \end{aligned}$$

Recall $z_{i,n}(\theta )=x_{i,n}^{*'}\beta +\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}y_{i,n}-\sigma _{ii,n}\frac{\phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}{1-\Phi (x_{i,n}^{*'}\beta /\sigma _{ii,n})}(1-y_{i,n})$, where $\sigma _{ii,n}=[(I_{n}-\rho W_{n})^{-1}(I_{n}-\rho W_{n})^{-1'}]_{ii}$. By the compactness of the parameter space and uniform boundedness of $X_{n}$, and $\sigma _{ii,n}^{2}=var(y_{i,n}^{*})$ is bounded away from 0, we have $\sup _{i,n,\theta \in \Theta }||\frac{\partial z_{i,n}(\theta )}{\partial \beta }||_{2}\equiv M_{1}<+\infty$ and $\sup _{i,n,\theta \in \Theta }|\frac{\partial z_{i,n}(\theta )}{\partial \ \rho }|\equiv M_{2}<+\infty$. Therefore,

$$\begin{aligned} \frac{1}{n}|\frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \beta ^{+}}|\le MM_{1}, \end{aligned}$$

and

$$\begin{aligned} \frac{1}{n}|\frac{\partial [\beta 'X_{n}'Z_{n}(\theta ^{+})]}{\partial \rho ^{+}}| \le MM_{2}. \end{aligned}$$

Then, $\frac{1}{n}||\frac{\partial [\bar{\beta }'X_{n}'Z_{n}(\bar{\theta }^{+})]}{\partial \gamma }||_{2}<+\infty$, so $\frac{1}{n}\beta 'X_{n}'Z_{n}(\theta ^{+})$ is stochastically equicontinous. In the similar ways as above, $\frac{1}{n}\rho \beta 'X_{n}'W_{n}Z_{n}(\theta ^{+})$ is are stochastically equicontinous.

Turning to the term of $\frac{1}{n}Z_{n}(\theta ^{+})'Z_{n}(\theta ^{+})$,

$$\begin{aligned} |\frac{1}{n}Z_{n}(\theta _{1}^{+})'Z_{n}(\theta _{1}^{+})-\frac{1}{n}Z_{n}(\theta _{2}^{+})'Z_{n}(\theta _{2}^{+})|&=\frac{1}{n}|\sum _{i=1}^{n}[z_{i,n}(\theta _{1}^{+})^{2}-z_{i,n}(\theta _{2}^{+})^{2}]|\\&=\frac{1}{n}|\sum _{i=1}^{n}z_{i,n}(\bar{\theta }^{+})\frac{\partial z_{i,n}(\bar{\theta }^{+})'}{\partial \theta ^{+}}(\theta _{1}^{+}-\theta _{2}^{+})|\\&\le z_{m}\frac{1}{n}\sum _{i=1}^{n}||\frac{\partial z_{i,n}(\bar{\theta }^{+})}{\partial \theta ^{+}}||_{2}||(\theta _{1}^{+}-\theta _{2}^{+})||_{2}, \end{aligned}$$

where $z_{m}\equiv \sup _{i,n,\theta \in \Theta }z_{i,n}(\theta )<+\infty$. As shown above,

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^{n}||\frac{\partial z_{i,n}(\bar{\theta }^{+})}{\partial \theta ^{+}}||_{2}<+\infty . \end{aligned}$$

Hence, $\frac{1}{n}Z_{n}(\theta ^{+})'Z_{n}(\theta ^{+})$ is stochastically equicontinous. $\frac{1}{n}\rho Z_{n}(\theta ^{+})W_{n}Z_{n}(\theta ^{+})$, $\frac{1}{n}\rho Z_{n}(\theta ^{+})W_{n}'Z_{n}(\theta ^{+})$ and $\frac{1}{n}\rho Z_{n}(\theta ^{+})W_{n}'W_{n}Z_{n}(\theta ^{+})$ can also be shown to be stochastically equicontinous using similar methods.

Therefore, $\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})$ is stochastically equicontinous. And

$$\begin{aligned} \sup _{(\theta ,\theta ^{+})\in \Theta \times \Theta }|\frac{1}{n}\tilde{Q}_{n}(\theta |\theta ^{+})-\tilde{Q}_{\infty }(\theta |\theta ^{+})|=0 \end{aligned}$$

by Theorem 1 in Donald (1992). $\square$