Semiparametric estimation in regression with missing covariates using single-index models

Abstract

We investigate semiparametric estimation of regression coefficients through generalized estimating equations with single-index models when some covariates are missing at random. Existing popular semiparametric estimators may run into difficulties when some selection probabilities are small or the dimension of the covariates is not low. We propose a new simple parameter estimator using a kernel-assisted estimator for the augmentation by a single-index model without using the inverse of selection probabilities. We show that under certain conditions the proposed estimator is as efficient as the existing methods based on standard kernel smoothing, which are often practically infeasible in the case of multiple covariates. A simulation study and a real data example are presented to illustrate the proposed method. The numerical results show that the proposed estimator avoids some numerical issues caused by estimated small selection probabilities that are needed in other estimators.

Appendix A

1.1 Regularity conditions

To establish the asymptotic theory in this work, we first assume the following general regularity conditions:

• (i) The smoothing parameter h satisfies $nh^2 \rightarrow \infty$ and $nh^{2r} \rightarrow 0$, as $n \rightarrow \infty$.

• (ii) All the selection probabilities $\pi _i$’s are bounded away from zero.

• (iii) The selection probability function on the single-index $\pi ^*({\gamma })$ has r continuous and bounded partial derivatives a.e.

• (iv) The density function f(u) of U and the conditional density function $f_{U|R}(u)$ of U|R have r continuous and bounded partial derivatives a.e.

• (v) The conditional distributions $f_{U|R=0}(u)$ and $f_{U|R=1}(u)$ have the same support, and $b(u) = f_{U|R=0}(u)/f_{U|R=1}(u)$ is bounded over the support.

• (vi) The conditional expectations ${\psi }(u| {\gamma }) = E(T| {Q}^\top {\gamma } = u)$ and $E(TT^\top | {Q}^\top {\gamma })$ exist and have r continuous and bounded partial derivatives a.e.

• (vii) For score T, $E(TT^\top )$ and $E\{(\partial /\partial {\beta })T\}$ exist and are positive definite, and $(\partial ^2/\partial {\beta } \partial {\beta }^\top )T$ exists and is continuous with respect to ${\beta }$ a.e.

1.2 Proof of Lemma 1

Proof

The idea in the proof is similar to that in the proof of Lemma 1 in Wang and Wang (2001). Recall that $u_i = Q_i^\top \gamma = y_i - \beta _Z^\top Z_i$ is the single index and that $n_1$ is the number of complete cases. Let

$$\begin{aligned} {\hat{f}}_{U|R=1}(u)= & {} \frac{1}{n_1 h} \sum \limits _{k=1}^n R_k K_h(u-u_k), \quad E_n(u) = {\hat{f}}_{U|R=1}(u) - f_{U|R=1}(u) ,\\ V_{ni}= & {} {\hat{f}}_{U|R=1}(u_i), \quad W_{ni} = \frac{1}{n_1 h} \sum \limits _{k=1}^n R_k T_{i,k} K_h(u_i - u_k) . \end{aligned}$$

Under the regularity conditions, we have $E\{E_n(u)\} = O(h^r)$ and $\mathrm{var}\{ E_n(u) \} = O\{(nh)^{-1}\}$ by the Taylor expansions. Then by the Chebyshev inequality, $E_n(u) - E\{E_n(u)\} = O_p \{ (nh)^{-1/2} \}$, which implies $E_n(u) = O_p\{ h^r + (nh)^{-1/2} \}$, and thus $E_n(u_i) = O_p\{ h^r + (nh)^{-1/2} \}$. Similarly, we have $W_{ni} - \psi _i V_{ni} = O_p \{ h^r + (nh)^{-1/2} \}$.

Define $\delta _n = h^{2r} + (nh)^{-1}$. Under the SIM condition,

$$\begin{aligned} {\hat{\psi }}_i - \psi _i= & {} \frac{W_{ni} - \psi _i V_{ni}}{f_{U|R=1}(u_i)} - \frac{(W_{ni} - \psi _i V_{ni})E_n(u_i)}{V_{ni} f_{U|R=1}(u_i)} \nonumber \\= & {} \frac{W_{ni} - \psi _i V_{ni}}{f_{U|R=1}(u_i)} + O_p(\delta _n). \end{aligned}$$

(A.1)

Let $Q_i^* = R_i Q_i$, $X_i^* = R_i X_i$ for $i=1,\ldots ,n$ as the values of the complete cases. Then

where $T^0_{i,k} = E_{Z_i|u_i,R_i=0}(T_{i,k})=\int T_{i,k} f(Z_i|u_i,R_i=0) \hbox {d}Z_i$, b(u) is defined in regularity condition (iv). The last step is because of the concentration of $u_i$ on $u_k$. Using the same idea and $\{\cdot \cdot \cdot \}$ to denote a repeat of the preceding term, we also have

Let

$$\begin{aligned} S_n = n^{-1/2} \sum \limits _{i=1}^n (1-R_i) \left\{ \frac{W_{ni} - \psi _i V_{ni}}{f_{U|R=1}(u_i)} - \frac{1}{n_1} \sum \limits _{k=1}^n R_k \left( T_k^0 - \psi _k^0\right) b(u_k) \right\} . \end{aligned}$$

Then the summations with $R_i=0$ in $S_n$ are i.i.d. random variables conditioning on all $(R,Q^*,X^*)$. Thus, we have

Then $E(S_n) = O(h^r)$ and $\mathrm{var}(S_n) = O(h^{2r} + (nh)^{-1})$ imply $S_n = O_p(\eta _n)$. Back to (A.1), we have

$$\begin{aligned} n^{-1/2} \sum \limits _{i=1}^n (1-R_i) \left( {\hat{\psi }}_i - \psi _i\right)&= n^{-1/2} \sum \limits _{i=1}^n \left\{ (1-R_i) \frac{1}{n_1} \sum \limits _{k=1}^n R_k \left( T_k^0 - \psi _k^0\right) b(u_k) \right\} + O_p(\eta _n) \\&= n^{-1/2} \sum \limits _{k=1}^n R_k \left( T_k^0 - \psi _k^0\right) a(u_k) + O_p(\eta _n). \end{aligned}$$

$\square$

1.3 Proof of Lemma 2

Proof

(a) The proof is analogous to that of Lemma 1. The main difference is that this is the summation of the complete cases. Thus we need to condition on $R_i = 1$. Then

where $T^1_{i,k} = E_{Z_i|u_i,R_i=1}(T_{i,k})=\int T_{i,k} f(Z_i|u_i,R_i=1) \hbox {d}Z_i$. The rest of the proof follows in the same manner as in the proof of Lemma 1.

(b) Similarly to the proof of (a), we have

$$\begin{aligned} n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i^*(\gamma )} \left\{ {\hat{\psi }}_i(\gamma )-\psi _i(\gamma )\right\} = n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i^*( \gamma )} \left\{ T_i^1 - \psi _i^1( \gamma )\right\} + O_p (\eta _n) . \end{aligned}$$

According to the Hölder inequality for the sum of the product terms in the second term below, we have

$$\begin{aligned}&n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{{\hat{\pi }}_i^*( \gamma )} \left\{ {\hat{\psi }}_i( \gamma )-\psi _i( \gamma )\right\} \\&\quad = n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i^*( \gamma )} \left\{ {\hat{\psi }}_i(\gamma )-\psi _i( \gamma )\right\} \\&\qquad + n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{{\hat{\pi }}_i^*( \gamma ) \pi _i^*(\gamma )} \{\pi _i^*(\gamma )-{\hat{\pi }}_i^*( \gamma )\} \left\{ {\hat{\psi }}_i( \gamma )-\psi _i( \gamma )\right\} \\&\quad = n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i^*( \gamma )} \left\{ T_i^1 - \psi _i^1( \gamma )\right\} + O_p (\eta _n) . \end{aligned}$$

(c) The proof can be obtained analogously as in (b). $\square$

1.4 Proof of Theorem 1

Proof

Based on the conclusion of Lemma 1,

$$\begin{aligned} \varDelta _3\left( \beta ,{\hat{\psi }}( \gamma )\right)&= n^{-1/2}\sum \limits _{i=1}^n R_i T_i + (1-R_i) \psi _i( \gamma ) + (1-R_i) \left\{ {\hat{\psi }}_i( \gamma ) - \psi _i(\gamma ) \right\} \\&= n^{-1/2}\sum \limits _{i=1}^n R_i T_i + (1-R_i) \psi _i( \gamma ) \\&\quad + R_i \left\{ T_i^0 - \psi _i^0(\gamma ) \right\} a\left( Q_i^\top \gamma \right) + O_p(\eta _n) \\&= n^{-1/2}\sum \limits _{i=1}^n U_i + O_p(\eta _n) . \end{aligned}$$

Since $\varDelta _3( \beta ,{\hat{\psi }}( \gamma ))$ is asymptotically equivalent to a sum of i.i.d. random variables, ${\hat{\beta }}_A$ is asymptotically normally distributed and has the asymptotic covariance ${\varvec{\varSigma }_A} = {{\varvec{D}}}^{-1} {\varvec{\mathcal {M}}} {{\varvec{D}}}^{-1}$ with

$$\begin{aligned} {\varvec{\mathcal {M}}}&= \mathrm{cov} \left( n^{-1/2}\sum \limits _{i=1}^n U_i \right) = \mathrm{cov} (U_1) \\&= \mathrm{cov} \left\{ R_1 T_1 + (1-R_1) \psi _i \right\} + \mathrm{cov} \left[ R_i \left\{ T_i^0 - \psi _i^0(\gamma ) \right\} a(Q_i^\top \gamma ) \right] \\&\quad + 2 \mathrm{cov} \left( R_1 T_1 + (1-R_1) \psi _i, R_i \left\{ T_i^0 - \psi _i^0(\gamma ) \right\} a\left( Q_i^\top \gamma \right) \right) \\&= {\varvec{\mathcal {A}}} + \varvec{{\mathcal {B}}} + 2{\varvec{\mathcal {C}}}. \end{aligned}$$

$\square$

1.5 Proof of Theorem 2

Proof

We first consider the first part, $\varDelta _1( \beta ,{\pi }(\hat{ \alpha }))$, of its estimating Eq. (11). By assumption, a correctly specified parametric model for the selection probabilities with parameter $\alpha$ is given by

$$\begin{aligned} \pi _i = \pi _i(\alpha ) = E(R_i| Q_i) = \pi ( \alpha | Q_i) . \end{aligned}$$

The log-likelihood is

$$\begin{aligned} l( \alpha ) = \sum \limits _{i=1}^n R_i \mathrm{log}\{\pi _i( \alpha )\} + (1-R_i)\mathrm{log}\{1-\pi _i( \alpha )\}. \end{aligned}$$

The corresponding estimating equation for MLE ${\hat{\alpha }}$ is given by

$$\begin{aligned} n^{-1/2}\sum \limits _{i=1}^n \frac{\pi _i'( \alpha )}{\pi _i( \alpha ) \{1-\pi _i( \alpha )\}} \{R_i - \pi _i( \alpha )\}=0. \end{aligned}$$

Then we have

$$\begin{aligned} n^{1/2}({\hat{\alpha }}- \alpha )= & {} \left[ E\left\{ \frac{\pi _1'( \alpha ) \pi _1'( \alpha )^\top }{\pi _1( \alpha ) \{1-\pi _1( \alpha )\}} \right\} \right] ^{-1} \\&\left\{ n^{-1/2}\sum \limits _{i=1}^n \frac{\pi _i'( \alpha )}{\pi _i( \alpha ) \{1-\pi _i( \alpha )\}} \{R_i - \pi _i( \alpha )\} \right\} + O_p(n^{-1/2}) . \end{aligned}$$

Moreover,

$$\begin{aligned} \varDelta _1( \beta ,{\pi }({\hat{\alpha }}))&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i({\hat{\alpha }})} T_i \\&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i( \alpha )} T_i - n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i^2( \alpha )} T_i \pi _i'( \alpha )^\top ({\hat{\alpha }}- \alpha ) + O_p(n^{-1/2}) \\&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i( \alpha )} T_i - E \left\{ \frac{1}{\pi _1( \alpha )} \psi _1 \pi _1'( \alpha )^\top \right\} n^{1/2} ({\hat{\alpha }}- \alpha ) + O_p(n^{-1/2}) \\&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i} T_i - E \left\{ \frac{1}{\pi _1( \alpha )} \psi _1 \pi _1'( \alpha )^\top \right\} \left[ E\left\{ \frac{\pi _1'( \alpha ) \pi _1'( \alpha )^\top }{\pi _1( \alpha ) \{1-\pi _1( \alpha )\}} \right\} \right] ^{-1} \\&\quad \left\{ n^{-1/2}\sum \limits _{i=1}^n \frac{\pi _i'( \alpha )}{\pi _i( \alpha ) \{1-\pi _i( \alpha )\}} \{R_i - \pi _i( \alpha )\} \right\} + O_p(n^{-1/2}) \\&= \varDelta _1( \beta ,\pi ) - {{\varvec{F}}}( \alpha ) {\varvec{C}}^{-1}( \alpha ) P_n( \alpha ) + O_p(n^{-1/2}) , \end{aligned}$$

where ${{\varvec{F}}}( \alpha )=E \left\{ \frac{1}{\pi _1( \alpha )} \psi _1 \pi _1'( \alpha )^\top \right\}$, ${{\varvec{C}}}( \alpha )=E\left\{ \frac{\pi _1'( \alpha ) \pi _1'( \alpha )^\top }{\pi _1( \alpha ) \{1-\pi _1( \alpha )\}} \right\}$, $P_n( \alpha ) = n^{-1/2}\sum \nolimits _{i=1}^n \frac{\pi _i'( \alpha )}{\pi _i( \alpha ) \{1-\pi _i( \alpha )\}} \{R_i$$- \pi _i( \alpha )\}$.

We now consider the second part of the estimating equation. By Lemmas 1 and 2(a), we obtain that

$$\begin{aligned} n^{-1/2}\sum \limits _{i=1}^n \left\{ {\hat{\psi }}_i( \gamma ) - \psi _i( \gamma ) \right\}&= n^{-1/2}\sum \limits _{i=1}^n R_i \left\{ {\hat{\psi }}_i( \gamma ) - \psi _i( \gamma ) \right\} \\&\quad + n^{-1/2}\sum \limits _{i=1}^n (1-R_i)\left\{ {\hat{\psi }}_i( \gamma ) - \psi _i( \gamma ) \right\} \\&= n^{-1/2} \sum \limits _{i=1}^n R_i \left\{ T_i^1 - \psi _i^1( \gamma )\right\} \\&\quad + n^{-1/2} \sum \limits _{i=1}^n R_i \left\{ T_i^0 - \psi _i^0( \gamma )\right\} a\left( Q_i^\top \gamma \right) + O_p (\eta _n) . \end{aligned}$$

Recall that the additional condition for Lemma 2(c) requires $\pi _i =\pi _i^*(\gamma )$. This implies that $T_i^0 = T_i^1 = E_{Z_i|u_i} (T_i)$, $\psi _i^0(\gamma ) = \psi _i^1(\gamma ) = E_{Z_i|u_i}\{\psi _i(\gamma )\}$. Let $T_i^* = E_{Z_i|u_i} (T_i)$, $\psi _i^*(\gamma ) = E_{Z_i|u_i}\{\psi _i(\gamma )\}$. Then

$$\begin{aligned} n^{-1/2}\sum \limits _{i=1}^n \left\{ {\hat{\psi }}_i( \gamma ) - \psi _i( \gamma ) \right\} = n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i}\left\{ T_i^* - \psi _i^*(\gamma ) \right\} + O_p(\eta _n) . \end{aligned}$$

(A.2)

Equation (A.2) and Lemma 2(c) imply that

$$\begin{aligned} n^{-1/2} \sum \limits _{i=1}^n \left\{ 1-\frac{R_i}{\pi _i(\hat{ \alpha })} \right\} \{{\hat{\psi }}_i( \gamma )-\psi _i( \gamma )\} = O_p (\eta _n) . \end{aligned}$$

Then

$$\begin{aligned} n^{-1/2} \sum \limits _{i=1}^n \left\{ 1-\frac{R_i}{\pi _i({\hat{\alpha }})} \right\} {\hat{\psi }}_i( \gamma ) = n^{-1/2} \sum \limits _{i=1}^n \left\{ 1-\frac{R_i}{\pi _i({\hat{\alpha }})} \right\} \psi _i( \gamma ) + O_p (\eta _n) . \end{aligned}$$

As in the proof for the first part $\varDelta _1( \beta ,{\pi }(\hat{ \alpha }))$, we can show that

$$\begin{aligned} n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i({\hat{\alpha }})} \psi _i( \gamma ) = n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i} \psi _i( \gamma ) - {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) P_n( \alpha ) + O_p(n^{-1/2}) . \end{aligned}$$

Finally we have

$$\begin{aligned} \varDelta _2\left( \beta ,{\pi }({\hat{\alpha }}),{\hat{\psi }}( \gamma )\right)&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i({\hat{\alpha }})} T_i + \left\{ 1-\frac{R_i}{\pi _i({\hat{\alpha }})} \right\} {\hat{\psi }}_i( \gamma ) \\&= n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i} T_i - {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) P_n( \alpha ) + n^{-1/2} \sum \limits _{i=1}^n \psi _i( \gamma ) \\&\quad - n^{-1/2}\sum \limits _{i=1}^n \frac{R_i}{\pi _i} \psi _i( \gamma ) + {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) P_n( \alpha ) + O_p (\eta _n) \\&= n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i} T_i + n^{-1/2} \sum \limits _{i=1}^n \left( 1-\frac{R_i}{\pi _i} \right) \psi _i( \gamma ) + O_p (\eta _n) \\&= n^{-1/2} \sum \limits _{i=1}^n \frac{R_i}{\pi _i^*( \gamma )} T_i + n^{-1/2} \sum \limits _{i=1}^n \left\{ 1-\frac{R_i}{\pi _i^*( \gamma )} \right\} \psi _i( \gamma ) + O_p (\eta _n) \\&= \varDelta _2\left( \beta ,\pi ^*( \gamma ),\psi \right) + O_p (\eta _n) . \end{aligned}$$

In summary, we have shown that $\varDelta _2( \beta ,{\pi }(\hat{ \alpha }),{\hat{\psi }}( \gamma ))$ is asymptotically equivalent to $\varDelta _2( \beta ,\pi ^*( \gamma ),\psi )$, which is a sum of i.i.d. terms. Hence, ${\hat{\beta }}_{\mathrm{PIP}A}$ is asymptotically equivalent to the solution of $\varDelta _2( \beta ,\pi ^*( \gamma ),\psi )=0$, having asymptotic normality with asymptotic covariance

$$\begin{aligned} {\varvec{\varSigma }}_{PA} = {{\varvec{D}}}^{-1}({\varvec{S}}-{{\varvec{S}}}^*+{{\varvec{V}}}){{\varvec{D}}}^{-1} . \end{aligned}$$

$\square$

1.6 Proof of Corollary 1

Proof

By the fact that

$$\begin{aligned} \varDelta _1\left( \beta ,{\pi }({\hat{\alpha }})\right) = \varDelta _1( \beta ,\pi ) - {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) P_n( \alpha ) + O_p(n^{-1/2}) , \end{aligned}$$

where ${{\varvec{F}}}( \alpha )$ and ${{\varvec{C}}}( \alpha )$ are given in the proof of Theorem 1, and by (A.1) in Wang et al. (1997), with an extension to a general parametric model, we have the asymptotic covariance for ${\hat{\beta }}_\mathrm{PIP}$ as

$$\begin{aligned} {\varvec{\varSigma }}_{P} = {{\varvec{D}}}^{-1}\left\{ \tilde{\varvec{S}}-{{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) {{\varvec{F}}}( \alpha )^\top \right\} {{\varvec{D}}}^{-1} , \end{aligned}$$

where $\tilde{{\varvec{S}}}=E(T_1 T_1^\top /\pi _1)$. By Wang and Wang (2001),

$$\begin{aligned} \tilde{\varvec{\varSigma }} = {{\varvec{D}}}^{-1}(\tilde{\varvec{S}}-\tilde{{\varvec{S}}}^*+{\varvec{V}}){{\varvec{D}}}^{-1} \end{aligned}$$

is the asymptotic covariance matrix for ${\hat{\beta }}$ when ${\hat{\psi }}$ is based on a standard kernel smoother, where $\tilde{{\varvec{S}}}^*=E(\psi _1 \psi _1^\top / \pi _1)$.

First we show that $\varvec{\varSigma }_P \succeq \tilde{\varvec{\varSigma }} .$ By the construction of the covariances, we only need to show that $\tilde{\varvec{S}}^*-{{\varvec{V}}} \succeq {{\varvec{F}}}( \alpha ) { \varvec{C}}^{-1}( \alpha ) {{\varvec{F}}}( \alpha )^\top$. Define $\xi = \left( \sqrt{\frac{1-\pi _1}{\pi _1}} \psi _1, \frac{\pi _1'( \alpha )}{\sqrt{(1-\pi _1)\pi _1}} \right) ^\top$. Then we have

$$\begin{aligned} E( \xi \xi ^\top )= & {} \left( {\begin{array}{cc} E\left( \frac{1-\pi _1}{\pi _1} \psi _1 \psi _1^\top \right) &{} E\left\{ \frac{1}{\pi _1} \psi _1 \pi _1'( \alpha )^\top \right\} \\ E\left\{ \frac{1}{\pi _1} \pi _1'( \alpha ) \psi _1^\top \right\} &{} E\left\{ \frac{\pi _1'( \alpha ) \pi _1'( \alpha )^\top }{(1-\pi _1)\pi _1} \right\} \\ \end{array} } \right) \\= & {} \left( {\begin{array}{cc} \tilde{{\varvec{S}}}^*-{{\varvec{V}}} &{} { {\varvec{F}}}( \alpha ) \\ {{\varvec{F}}}( \alpha )^\top &{} {{\varvec{C}}}( \alpha ) \\ \end{array} } \right) \succeq 0. \end{aligned}$$

By the Schur complement condition of the matrix above, we have

$$\begin{aligned} (\tilde{{\varvec{S}}}^*-{{\varvec{V}}}) - {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) {{\varvec{F}}}( \alpha )^\top \succeq 0 . \end{aligned}$$

Therefore, $\tilde{{\varvec{S}}}^*-{{\varvec{V}}} \succeq {{\varvec{F}}}( \alpha ) {{\varvec{C}}}^{-1}( \alpha ) {{\varvec{F}}}( \alpha )^\top$, which implies that ${\varvec{\varSigma }}_P \succeq \tilde{\varvec{\varSigma }}$.

Next, we show that $\tilde{\varvec{\varSigma }} = \varvec{\varSigma }_{A} = \varvec{\varSigma }_{PA}$ and thus the asymptotic equivalence between ${\hat{\beta }}_A$ and ${\hat{\beta }}_{\mathrm{PIP}A}$. Based on the results of Theorem 1, we can rewrite $\varDelta _3( \beta ,{\hat{\psi }}( \gamma ))$ as

$$\begin{aligned} \varDelta _3\left( \beta ,{\hat{\psi }}( \gamma )\right)&= n^{-1/2} \sum \limits _{i=1}^n U_i +O_p(\eta _n) \\&= n^{-1/2} \sum \limits _{i=1}^n \left[ \frac{R_i}{\pi _i^*(\gamma )} T_i + \left\{ 1-\frac{R_i}{\pi _i^*(\gamma )} \right\} \psi _i \right. \\&\quad \left. + R_i a(Q_i^\top \gamma ) \left\{ (T_i^0 - \psi _i^0) - (T_i - \psi _i) \right\} \right] +O_p(\eta _n) . \end{aligned}$$

The condition $E(Z_i|u_i) = Z_i$ implies that $T_i^0 = T_i^1 = T_i$ and $\psi _i^0(\gamma ) = \psi _i^1(\gamma ) = \psi _i(\gamma )$. By Theorem 2, both $\varDelta _2( \beta ,{\pi }(\hat{ \alpha }),{\hat{\psi }}( \gamma ))$ and $\varDelta _3( \beta ,{\hat{\psi }}( \gamma ))$ are asymptotically equivalent to $\varDelta _2( \beta ,\pi ^*( \gamma ),\psi )$ and thus have the same asymptotic covariance matrix as

$$\begin{aligned} \varvec{\varSigma }_{A} = \varvec{\varSigma }_{PA} = {\varvec{D}}^{-1}({{\varvec{S}}}-{{\varvec{S}}}^*+{\varvec{V}}){{\varvec{D}}}^{-1}. \end{aligned}$$

Recall the condition of Lemma 2(c) that $\pi _i = \pi _i^*(\gamma )$. Then ${{\varvec{S}}} = \tilde{{\varvec{S}}}$, ${{\varvec{S}}}^* = \tilde{{\varvec{S}}}^*$. Thus, we finally have

$$\begin{aligned} {\varvec{\varSigma }}_P \succeq \tilde{\varvec{\varSigma }} = {\varvec{\varSigma }_A} = {\varvec{\varSigma }_{PA}}. \end{aligned}$$

$\square$