Empirical likelihood-based weighted rank regression with missing covariates

Abstract

This paper proposes an empirical likelihood-based weighted (ELW) rank regression approach for estimating linear regression models when some covariates are missing at random. The proposed ELW estimator of regression parameters is computationally simple and achieves better efficiency than the inverse probability weighted (IPW) estimator if the probability of missingness is correctly specified. The covariances of the IPW and ELW estimators are estimated by using a variant of the induced smoothing method, which can bypass density estimation of the errors. Simulation results show that the ELW method works well in finite samples. A real data example is used to illustrate the proposed ELW method.

Appendix

Unless mentioned otherwise, all limits are taken as $n\rightarrow \infty$ and $\Vert \cdot \Vert$ denotes the Euclidean norm. For notational convenience, for $i=1\ldots ,n$, let $U_{Bi}=U_{Bi}(\gamma ^*)$, $h_i=h_i(\alpha ^*,\beta ^*,\gamma ^*)$ and $g_i=g_i(\alpha ^*,\beta ^*,\gamma ^*)$. Write

$$\begin{aligned}&G(\lambda ,\alpha ,\beta ,\gamma )=\frac{1}{n}\sum _{i=1}^n\frac{g_i(\alpha ,\beta ,\gamma )}{1+\lambda ^\textsf {T}g_i(\alpha ,\beta ,\gamma )},\\&U_g=\frac{1}{n}\sum _{i=1}^ng_i,\ \ U_h=\frac{1}{n}\sum _{i=1}^nh_{i},\ \ U_B=\frac{1}{n}\sum _{i=1}^nU_{Bi},\\&G_\gamma =\lim _{n\rightarrow \infty }n^{-1}\sum _{i=1}^n\left\{ \frac{\partial g_i(\alpha ^*,\beta ^*,\gamma ^*)}{\partial \gamma ^\textsf {T}}\right\} . \end{aligned}$$

To establish all the large sample properties in this paper, we require the following conditions:

Regularity conditions

• C1: $\{(y_i,x_i,z_i,\delta _i)\}_{i=1}^n$ are independent.

• C2: The parameter $\beta ^*$ is an interior point of a compact parameter space $\varTheta \subset \mathcal {R}^p$.

• C3: $w_i$ has a bounded support.

• C4: For $i,j=1,\ldots ,n$, let $f_{i}$ and $f_{ij}$ denote respectively the condition density functions of $e_i$ and $e_i-e_j$ given $(w_i,w_j)$. Assume that $f_{i}'(\cdot )$ exists and is uniformly bounded and $f_{ij}$ satisfies the following assumptions:

  • (1) $f_{ij}(-u)=f_{ij}(u)$, $u\in \mathcal {R}$.

  • (2) If $0\le u<v$, then $f_{ij}(u)\ge f_{ij}(v)$.

  • (3) There exists a $\varDelta >0$, when $0\le u<v<\varDelta$, $f_{ij}(u)>f_{ij}(v)$.

  • (4) For fixed $t \in \mathcal {R}$, $\int _{-\infty }^{\infty }|u+t|f_{ij}(u)du<\infty$.

• C5: A, $S_\varphi$, $S_B$, $S_{g}$, $S_\varphi -F_\gamma S_B^{-1}F_\gamma ^\textsf {T}$ and $S_\varphi -F_gS_g^{-1}F_g^\textsf {T}$ are positive definite.

• C6: (a) For all $(y_{i},z_{i})$, $\pi (y_{i},z_{i},\gamma )$ admits all third partial derivatives $\frac{\partial ^3\pi (y_{i},z_{i},\gamma )}{\partial \gamma _k\partial \gamma _l\partial \gamma _m}$ for all $\gamma$ in a neighborhood of the true value $\gamma ^*$, $\max _{1\le i \le n }\biggr \Vert \frac{\partial ^3\pi (y_{i},z_{i},\gamma )}{\partial \gamma _k\partial \gamma _l\partial \gamma _m}\biggr \Vert$ is bounded by an integrable function for all $\gamma$ in this neighborhood, and $\max _{1\le i \le n }\Vert \partial \pi (y_i,z_i,\gamma )/\partial \gamma \Vert ^2$ is bounded by an integrable function for all $\gamma$ in this neighborhood.

  (b) The probability $\pi (y,z,\gamma ^*)$ is bounded away from zero, i.e. $\inf _{(y,z)}\pi (y,z,\gamma ^*) \ge c_0$ for some $c_0>0.$

• C7: $\max _{1\le i \le n }\Vert \xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )\Vert ^2$ is bounded by an integrable function for all $\eta$ in a neighborhood of $\eta ^*$ and $\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )$ is continuous at each $\eta$ with probability one in this neighborhood, where $\eta =(\alpha ^\textsf {T},\beta ^\textsf {T})^\textsf {T}$ and $\eta ^*=(\alpha ^{*\textsf {T}},\beta ^{*\textsf {T}})^\textsf {T}$. Moreover,

  $$\begin{aligned} \sup _{\Vert \eta -\eta ^*\Vert \le cn^{-1/2}}\left\| n^{-1/2}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\{\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )-\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta ^*)\}\right\| =o_p(1). \end{aligned}$$

Most of the above conditions were assumed for a standard rank regression model. Additional conditions are on the missing data mechanism and the unconditional moment restrictions. C1 defines the structure which generates the observations. C2 is a standard assumption for a parameter space. C3–C4 impose some conditions on the conditional error distributions and the covariates, both of which will hold in most practical situations. C5 guarantees that the asymptotic covariance matrices of the IPW and ELW estimators are both positive definite. C6(a) contains the conditions, which are needed to establish the consistency and asymptotic normality of the binomial likelihood estimator $\hat{\gamma }$. C6(b) implies that the covariates x cannot be missing with probability 1 anywhere in the domain of the (y, z). C7 collects the conditions on $h(t,\alpha ,\beta ,\gamma )$ under which the influence function of the multiplier $\hat{\lambda }$ can be established.

In the following, we show that the regularity condition C4 is easily satisfied.

Lemma A.1

Consider the conditions:

• (a) $f_1=\ldots =f_n=f$, f is unimodal;

• (b) $f_i$ is symmetrical unimodal and has a unique mode at $\delta$, $i=1,\ldots ,n$;

• (c) For fixed $t \in \mathcal {R}$, $\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }|u-v+t|f_{i}(u)f_{j}(v)dudv<\infty$.

Then either (a) or (b) together with (c) imply the condition C4.

Proof of Lemma A.1

Without loss of generality, we let $\delta =0$ for (b).

$$\begin{aligned} f_{ij}(u)= & {} \int _{-\infty }^{\infty } f_i(u+v)f_j(v)dv. \end{aligned}$$

By Theorems 2.1 and 2.2 in Purkayastha (1998), we get that either (a) or (b) yields the points (1) and (2) of condition C4.

Let $J(u)=f_{ij}(u)-f_{ij}(0).$ It follows that J(u) is symmetrical about 0 and $J(u)\le 0$ for all u. Thus we only show that $J(u)<0$ for $u>0$.

If (a) is satisfied, by the Cauchy–Schwarz inequality,

$$\begin{aligned} f_{ij}(u)= & {} \int _{-\infty }^{\infty } f_i(u+v)f_j(v)dv=\int _{-\infty }^{\infty } f(u+v)f(v)dv\\\le & {} \sqrt{ \int _{-\infty }^{\infty } f^2(u+v)dv}\sqrt{ \int _{-\infty }^{\infty } f^2(v)dv}\\= & {} \int _{-\infty }^{\infty } f^2(v)dv=f_{ij}(0), \end{aligned}$$

and equality holds if and only if $f(u+v)=cf(u)$. Since $\int _{-\infty }^{\infty }f(u+v)dv=c \int _{-\infty }^{\infty }f(v)dv=1.$ We have $c=1.$ Thus, $J(u)<0$ for $u>0$.

If (b) is satisfied, we decompose $J(u)=J_1(u)+J_2(u),$ where

$$\begin{aligned} J_1(u)= & {} \int _{0}^{\infty } \{f_i(u+v)-f_i(v)\}\{f_j(v)-f_j(u+v)\}dv, \end{aligned}$$

(A.1)

and

$$\begin{aligned} J_2(u)= & {} \int _{0}^{\infty } \{f_i(u+v)-f_i(v)\}f_j(u+v)dv+\int _{-\infty }^{0} \{f_i(u+v)-f_i(v)\}f_j(v)dv. \end{aligned}$$

By the substitution $t=-(u+v)$ in the integral, we see that

$$\begin{aligned} \int _{-\infty }^{0} \{f_i(u+v)-f_i(v)\}f_j(v)dv= & {} \int _{-u}^{\infty }\{f_i(t)-f_i(t+u)\}f_j(t+u)dt . \end{aligned}$$

Then

$$\begin{aligned} J_2(u)= & {} \int _{-u}^{0}\{f_i(t)-f_i(t+u)\}f_j(t+u)dt. \end{aligned}$$

By using the substitution $t=v+\frac{u}{2}$, we obtain

$$\begin{aligned} J_2(u)= & {} \int _{-u/2}^{u/2}\left\{ f_i\left( v-\frac{u}{2}\right) -f_i\left( v+\frac{u}{2}\right) \right\} f_j\left( v+\frac{u}{2}\right) dv\\= & {} \int _{-u/2}^{0}\left\{ f_i\left( v-\frac{u}{2}\right) -f_i\left( v+\frac{u}{2}\right) \right\} f_j\left( v+\frac{u}{2}\right) dv\\&+\int _{0}^{u/2}\left\{ f_i\left( v-\frac{u}{2}\right) -f_i\left( v+\frac{u}{2}\right) \right\} f_j\left( v+\frac{u}{2}\right) dv. \end{aligned}$$

For the first integral above, substitute $t=-v$,

$$\begin{aligned}&\int _{-u/2}^{0}\left\{ f_i\left( v-\frac{u}{2}\right) -f_i\left( v+\frac{u}{2}\right) \right\} f_j\left( v+\frac{u}{2}\right) dv = \int _{0}^{u/2}\left\{ f_i\left( t+\frac{u}{2}\right) \right. \\&\quad \left. -f_i\left( t-\frac{u}{2}\right) \right\} f_j\left( t-\frac{u}{2}\right) dt. \end{aligned}$$

Thus,

$$\begin{aligned} J_2(u)= & {} \int _{0}^{u/2}\left\{ f_i\left( v-\frac{u}{2}\right) -f_i\left( v+\frac{u}{2}\right) \right\} \left\{ f_j\left( v+\frac{u}{2}\right) -f_j\left( v-\frac{u}{2}\right) \right\} dv.\nonumber \\ \end{aligned}$$

(A.2)

Under (b), it is straightforward to show that there exists a $\delta _1>0$, when $0\le u<v<\delta _1$, $f_{i}(u)>f_{i}(v)$, $i=1,\ldots ,n$. Then (A.1) and (A.2) imply that $J(u)<0$ for $u>0$.

Then we have if either (a) or (b) is satisfied, $f_{ij}(u)$ has a unique maximum at 0, which yields the point (3) of condition C4. By the variable substitution, we have

$$\begin{aligned} \int _{-\infty }^{\infty }|u+t|f_{ij}(u)du= & {} \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }|u+t|f_i(u+v)f_j(v)dv du\\= & {} \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }|u-v+t|f_i(u)f_j(v)dv du. \end{aligned}$$

Then, the point (4) of condition C4 follows from (c). $\square$

Lemma A.2

Suppose that the regularity conditions C1–C4 are satisfied, then

$$\begin{aligned} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^nE\{|e_i(\beta )-e_j(\beta )|\} \end{aligned}$$

has the unique minimum at $\beta ^*$.

Proof of Lemma A.2

Observe that

$$\begin{aligned}&\frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^nE\{|e_i(\beta )-e_j(\beta )|\}\\&\quad =\frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n E\{|e_i-e_j+(w_i-w_j)^\textsf {T}(\beta ^*-\beta )|\}\\&\quad =\frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-\infty }^{\infty }|u+t_{ij}|f_{ij}(u)du, \end{aligned}$$

where $t_{ij}=(w_i-w_j)^\textsf {T}(\beta ^*-\beta )$. Let

$$\begin{aligned} J^*(t)=\frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-\infty }^{\infty }\{|u+t|-|u|\}f_{ij}(u)du. \end{aligned}$$

Note that for $t>0$,

$$\begin{aligned} J^*(t)= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-\infty }^{-t}\{-u-t+u\}f_{ij}(u)du\\&+\, \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-t}^{0}\{u+t+u\}f_{ij}(u)du\\&+\, \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{0}^{\infty }\{u+t-u\}f_{ij}(u)du\\= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\left\{ -t\int _{-\infty }^{-t} f_{ij}(u)du+t\int _{-t}^{0}f_{ij}(u)du+2\int _{-t}^{0}uf_{ij}(u)du\right. \\&\left. +\, t\int _{0}^{\infty }f_{ij}(u)du\right\} \\= & {} \frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n \int _{0}^{t}\{t-u\} f_{ij}(u)du>0. \end{aligned}$$

Similarly, for $t<0$,

$$\begin{aligned} J^*(t)= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-\infty }^{0}\{-u-t+u\}f_{ij}(u)du\\&+\, \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{0}^{-t}\{-u-t-u\}f_{ij}(u)du\\&+\, \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\int _{-t}^{\infty }\{u+t-u\}f_{ij}(u)du\\= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\left\{ -t\int _{-\infty }^{0} f_{ij}(u)du-t\int _{0}^{-t}f_{ij}(u)du-2\int _{0}^{-t}uf_{ij}(u)du\right. \\&\left. +\, t\int _{-t}^{\infty }f_{ij}(u)du\right\} \\= & {} \frac{2}{n^2}\sum _{i=1}^n\sum _{j=1}^n \int _{0}^{-t}\{-t-u\} f_{ij}(u)du>0. \end{aligned}$$

Therefore, we have $J^*(t)\ge 0$ for all t and equality holds if and only if $t=0.$ Lemma A.2 is then proved. $\square$

Proof of Theorem 1

Let $\hat{F}(\beta ,t)=\frac{1}{n} \sum _{i=1}^n\frac{\delta _i}{\pi (y_i,z_i,\gamma ^*)} I(e_i(\beta )\le t)$, $F_n(\beta ,t)=\frac{1}{n}\sum _{i=1}^{n}F_i(t+w_i^\textsf {T}(\beta -\beta ^*))$. Then we have

$$\begin{aligned} L_{1n}(\beta )= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n\frac{\delta _i\delta _j}{\pi (y_i,z_i,\hat{\gamma })\pi (y_j,z_j,\hat{\gamma })} |e_i(\beta )-e_j(\beta )|\\= & {} \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }|u-v|d\hat{F}(\beta ,u)d\hat{F}(\beta ,v)+o_{a.s.}(1). \end{aligned}$$

The uniform strong law of large numbers (Pollard 1990, p. 41) can be easily adapted to give

$$\begin{aligned} \sup _{t,\beta }|\hat{F}(\beta ,t)-F_n(\beta ,t) |\rightarrow 0, \end{aligned}$$

Thus $L_{1n}(\beta )$ converges uniformly to

$$\begin{aligned} L(\beta )= & {} \lim _{n\rightarrow \infty } \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }|u-v|d F_n(\beta ,u)d F_n(\beta ,v). \end{aligned}$$

By Lemma A.2, it can be showed that $L(\beta )$ has a unique minimizer $\beta =\beta ^*$. Hence, $\hat{\beta }_{IPW}$, as minimizer of $L_{1n}(\beta )$, converges to $\beta ^*$ almost surely. $\square$

Proof of Theorem 2

It follows from directional differentiation that minimizing $L_{1n}(\beta )$ is equivalent to solving $U_{1n}(\beta )=0$, where

$$\begin{aligned} U_{1n}(\beta )= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n \frac{\delta _i\delta _j(w_i-w_j)}{\pi (y_i,z_i,\hat{\gamma })\pi (y_j,z_j,\hat{\gamma })} \text{ sgn }(e_i(\beta )-e_j(\beta )). \end{aligned}$$

Define

$$\begin{aligned} U_n^*(\beta )= & {} \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n \frac{\delta _i\delta _j(w_i-w_j)}{\pi (y_i,z_i,\gamma ^*)\pi (y_j,z_j,\gamma ^*)} \text{ sgn }(e_i(\beta )-e_j(\beta )). \end{aligned}$$

The proof is divided into two steps. In the first step, we show that $n^{1/2}U_n^*(\beta ^*){\mathop {\longrightarrow }\limits ^{d}}N(0,V)$. Let $\hat{G}(\beta ,t)=\frac{1}{n} \sum _{i=1}^n\frac{\delta _i w_i}{\pi (y_i,z_i,\gamma ^*)} I(e_i(\beta )\le t)$, $G_n(\beta ,t)=\frac{1}{n}\sum _{i=1}^{n}w_iF_i(t+w_i^\textsf {T}(\beta -\beta ^*))$. Then we have

$$\begin{aligned} U_n^*(\beta )= & {} \int _{-\infty }^{\infty }\int _{-\infty }^{\infty } \text{ sgn }(u-v)d \hat{F}(\beta ,v) d\hat{G}(\beta ,u) \nonumber \\&-\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d \hat{F}(\beta ,u)d \hat{G}(\beta ,v) \nonumber \\= & {} 2\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d \hat{F}(\beta ,v)d \hat{G}(\beta ,u). \end{aligned}$$

(A.3)

Write

$$\begin{aligned} n^{1/2}U_n^*(\beta )= & {} 2n^{1/2}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d \{\hat{F}(\beta ,v)-F_n(\beta ,v)\} d \hat{G}(\beta ,u) \nonumber \\&+\, 2n^{1/2}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d F_n(\beta ,v) d \hat{G}(\beta ,u). \end{aligned}$$

(A.4)

It can easily be shown that $n^{1/2}\{\hat{F}(\beta ,\cdot )-F_n(\beta ,\cdot )\}$ converges weakly to a Gaussian process. Thus

$$\begin{aligned}&n^{1/2}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d \{\hat{F}(\beta ,v)-F_n(\beta ,v)\} d \hat{G}(\beta ,u) \nonumber \\&\quad = n^{1/2}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d \{\hat{F}(\beta ,v)-F_n(\beta ,v)\} d G_n(\beta ,u)+o_p(1) \nonumber \\&\quad =n^{1/2}\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d\hat{F}(\beta ,v)d G_n(\beta ,u)+o_p(1), \end{aligned}$$

(A.5)

where the last equality follows from

$$\begin{aligned}&\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d F_n(\beta ,v) d G_n(\beta ,u)\\&\quad = \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n w_i \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v) f_i(u)f_j(v)dudv\\&\quad = \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n w_i \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u)f_i(u+v)f_j(v)dudv\\&\quad = \frac{1}{n^2}\sum _{i=1}^n\sum _{j=1}^n w_i \int _{-\infty }^{\infty } \text{ sgn }(u)f_{ij}(u)du=0. \end{aligned}$$

Combining (A.4) with (A.5) we obtain $n^{1/2}U_n^*(\beta ^*)=n^{-1/2}\sum _{i=1}^n\psi _i^*+o_p(1)$, where

$$\begin{aligned} \psi _i^*= & {} \frac{2\delta _i}{\pi (y_i,z_i,\gamma ^*)}\left\{ w_i\int _{-\infty }^{\infty }\text{ sgn }(e_i-v)dF_n(\beta ^*,v)+\int _{-\infty }^{\infty }\text{ sgn }(u-e_i)dG_n(\beta ^*,u)\right\} \nonumber \\= & {} \frac{2\delta _i}{\pi (y_i,z_i,\gamma ^*)}\left[ w_i\{2F_n(\beta ^*,e_i)-1\}+\{\bar{w}-2G_n(\beta ^*,e_i)\}\right] \end{aligned}$$

(A.6)

and $\bar{w}=n^{-1}\sum _{j=1}^n w_j$. Thus, $n^{1/2}U_n^*(\beta ^*)$ converges in distribution to N(0, V) by the multivariate central limit theorem, where

$$\begin{aligned} V=\lim _{n\rightarrow \infty }n^{-1}\sum _{i=1}^n\psi _i^*\psi _i^{*\textsf {T}}. \end{aligned}$$

In the second step, we show that

$$\begin{aligned} n^{1/2}U_n^*(\beta )-n^{1/2}U_n^*(\beta ^*)=A_n n^{1/2}(\beta -\beta ^*)+o_p(1+n^{1/2}\Vert \beta -\beta ^*\Vert ), \end{aligned}$$

where

$$\begin{aligned}&A_{n}=2\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)dF_n(\beta ^*,v)d A_{n,G}(u)\\&\qquad +\, 2\int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\text{ sgn }(u-v)d A_{n,F}(v)dG_n^\textsf {T}(\beta ^*,u) \\&\quad =2n^{-2}\sum _{i=1}^n\sum _{j=1}^n(w_i-w_j)(w_i-w_j)^\textsf {T}\int _{-\infty }^{\infty }f_i(u)d F_j(u), \end{aligned}$$

with

$$\begin{aligned} A_{n,F}(t)=\frac{1}{n}\sum ^n_{i=1}w_if_i(t)\ \ \text{ and }\ \ A_{n,G}(t)=\frac{1}{n}\sum ^n_{i=1}w_i w_i^\textsf {T}f_i(t). \end{aligned}$$

Regarding $\hat{F}$ and $\hat{G}$ as sums, over i, of independent random variables, we can apply the approximations given in Lai and Ying (1988) for weighted empirical processes to obtain

$$\begin{aligned}&\sup _{t,\Vert \beta -\beta ^*\Vert \le d_n}\left\{ |\hat{F}(\beta ,t)-\hat{F}(\beta ^*,t)-A_{n,F}(t)(\beta -\beta ^*)|/(n^{-1/2}+\Vert \beta -\beta ^*\Vert ) \right\} =o_p(1),\nonumber \\ \end{aligned}$$

(A.7)

$$\begin{aligned}&\sup _{t,\Vert \beta -\beta ^*\Vert \le d_n}\left\{ |\hat{G}(\beta ,t)-\hat{G}(\beta ^*,t)-A_{n,G}(t)(\beta -\beta ^*)|/(n^{-1/2}+\Vert \beta -\beta ^*\Vert ) \right\} =o_p(1),\nonumber \\ \end{aligned}$$

(A.8)

where $d_n\rightarrow 0.$ (A.3), (A.7) and (A.8) imply that

$$\begin{aligned} n^{1/2}U_n^*(\beta )-n^{1/2}U_n^*(\beta ^*)=A_n n^{1/2}(\beta -\beta ^*)+o_p(1+n^{1/2}\Vert \beta -\beta ^*\Vert ).\quad \end{aligned}$$

(A.9)

The asymptotic linearity of $U_n^*$ in (A.9) now gives

$$\begin{aligned} n^{1/2}(\hat{\beta }_{IPW}-\beta ^*)=A_n^{-1}\{n^{1/2}U_n^*(\hat{\beta }_{IPW})-n^{1/2}U_n^*(\beta ^*)\}+o_p(1). \end{aligned}$$

(A.10)

Moreover,

$$\begin{aligned} 0\doteq U_{1n}(\hat{\beta }_{IPW})= & {} U_n^*(\hat{\beta }_{IPW})+F_\gamma (\hat{\gamma }-\gamma )+o_p(n^{-1/2}) \nonumber \\= & {} U_n^*(\hat{\beta }_{IPW})+F_\gamma S_B^{-1}U_B+o_p(n^{-1/2}). \end{aligned}$$

(A.11)

where

$$\begin{aligned} F_\gamma= & {} -\lim _{n\rightarrow \infty }2n^{-2}\sum _{i=1}^n\sum _{j=1}^n\frac{\delta _i\delta _j(w_i-w_j)\partial \pi (y_i,z_i,\gamma ^*)/\partial \gamma ^\textsf {T}}{\pi ^2(y_i,z_i,\gamma ^*)\pi (y_j,z_j,\gamma ^*)}\text{ sgn }(e_i-e_j)\\= & {} -\lim _{n\rightarrow \infty }2n^{-2}\sum _{i=1}^n\sum _{j=1}^n(w_i-w_j)\text{ sgn }(e_i-e_j)\frac{\partial \pi (y_i,z_i,\gamma ^*)/\partial \gamma ^\textsf {T}}{\pi (y_i,z_i,\gamma ^*)}. \end{aligned}$$

(A.10) and (A.11) lead to the following asymptotic expansion

$$\begin{aligned} n^{1/2}(\hat{\beta }_{IPW}-\beta ^*)=-A_n^{-1}\{n^{1/2}U_n^*(\beta ^*)+n^{1/2}F_\gamma S_B^{-1}U_B\}+o_p(1). \end{aligned}$$

Then, we have $n^{1/2}(\hat{\beta }_{IPW}-\beta ^*)$ converges to $N(0,\varSigma _{IPW})$, where $\varSigma _{IPW}=A^{-1}VA^{-1}$, $A=\lim _{n\rightarrow \infty }A_n$ and $V=\lim _{n\rightarrow \infty }\text{ var }(n^{1/2}U_n^*(\beta ^*)+n^{1/2}F_\gamma S_B^{-1}U_B)=S_\varphi -F_\gamma S_B^{-1}F_\gamma ^\textsf {T}$. Then the proof is completed. $\square$

Lemma A.3

If $\lambda =\lambda (\theta )$ solves

$$\begin{aligned} \sum _{i=1}^n\frac{g_i(\theta )}{1+\lambda ^\textsf {T} g_i(\theta )}=0, \end{aligned}$$

(A.12)

then we have $\Vert \lambda (\theta )\Vert =O_p(n^{-1/2})$ and

$$\begin{aligned} \lambda (\theta )= & {} \left\{ \frac{1}{n}\sum _{i=1}^n g_i(\theta )g_i^\textsf {T}(\theta )\right\} ^{-1} \frac{1}{n}\sum _{i=1}^n g_i(\theta )+o_p\left( n^{-1/2}\right) . \end{aligned}$$

uniformly about $\theta \in B_0=\{\theta :\Vert \theta -\theta ^*\Vert \le c n^{-1/2}\}$ for some $0< c<\infty$.

Proof of Lemma A.3

The basic idea behind this proof is outlined in Owen (1990). We first show that

$$\begin{aligned} \sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n g_i(\theta )\right\| =O_p\big (n^{-1/2}\big ). \end{aligned}$$

(A.13)

From the definition of g, we only need to show $\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n h_i(\theta )\right\| =O_p(n^{-1/2})$ and $\sup _{\Vert \gamma -\gamma ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^nU_{Bi}(\gamma )\right\| =O_p(n^{-1/2})$. On one hand, by C7 and

$$\begin{aligned} \frac{1}{n}\sum _{i=1}^nU_{Bi}(\gamma )=\frac{1}{n}\sum _{i=1}^nU_{Bi}(\gamma ^*)+\frac{1}{n}\sum _{i=1}^n\frac{ \partial U_{Bi}(\bar{\gamma })}{\partial \gamma ^\textsf {T}}(\gamma -\gamma ^*), \end{aligned}$$

where $\bar{\gamma }$ is a point on the segment connecting $\gamma$ and $\gamma ^*$, $\sup _{\Vert \gamma -\gamma ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\right. \left. U_{Bi}(\gamma )\right\| =O_p(n^{-1/2})$ is proved. On the other hand,

$$\begin{aligned}&\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n h_i(\theta )\right\| \\&\quad =\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma )}{\pi (y_i,z_i,\gamma )}\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )\right\| \\&\quad \le \sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\left\{ \frac{\delta _i-\pi (y_i,z_i,\gamma )}{\pi (y_i,z_i,\gamma )}-\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\right\} \xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )\right\| \\&\qquad +\left\| \frac{1}{n}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\xi (\mathcal {Y}_n,\mathcal {Z}_n,\eta ^*)\right\| \\&\qquad +\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\{\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )-\xi (\mathcal {Y}_n,\mathcal {Z}_n,\eta ^*)\}\right\| \\&\quad =\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\{\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )-\xi (\mathcal {Y}_n,\mathcal {Z}_n,\eta ^*)\}\right\| \\&\qquad +O_p(n^{-1/2})\\&\quad \le \sup _{\Vert \eta -\eta ^*\Vert \le cn^{-1/2}}\left\| \frac{1}{n}\sum _{i=1}^n\frac{\delta _i-\pi (y_i,z_i,\gamma ^*)}{\pi (y_i,z_i,\gamma ^*)}\{\xi _{i}(\mathcal {Y}_n,\mathcal {Z}_n,\eta )-\xi (\mathcal {Y}_n,\mathcal {Z}_n,\eta ^*)\}\right\| \\&\qquad +O_p\big (n^{-1/2}\big )\\&\quad =o_p\big (n^{-1/2}\big )+O_p\big (n^{-1/2}\big )\\&\quad =O_p\big (n^{-1/2}\big ). \end{aligned}$$

Let $U_i=\lambda ^\textsf {T} g_i(\theta )$ and $g^*=\max _{1\le i \le n}\sup _{\theta \in B_0}\Vert g_i(\theta )\Vert$. Let $\lambda (\theta )=\Vert \lambda (\theta )\Vert v$, $\Vert v\Vert =1$. Substituting $1/(1+U_i)=1-U_i/(1+U_i)$ into (A.12) and simplifying, we find that

$$\begin{aligned} \Vert \lambda (\theta )\Vert v^\textsf {T}\frac{1}{n}\sum _{i=1}^n\frac{g_i(\theta )g_i^\textsf {T}(\theta )}{1+U_i}v=v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta ). \end{aligned}$$

Since every $p_i>0$, we have $1+U_i>0$ and therefore

$$\begin{aligned} \Vert \lambda (\theta )\Vert v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta )g_i^\textsf {T}(\theta )v\le & {} \Vert \lambda (\theta )\Vert v^\textsf {T}\frac{1}{n}\sum _{i=1}^n\frac{g_i(\theta )g_i^\textsf {T}(\theta )}{1+U_i}v(1+\Vert \lambda (\theta )\Vert g^*)\\= & {} v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta )(1+\Vert \lambda (\theta )\Vert g^*). \end{aligned}$$

Consequently,

$$\begin{aligned} \Vert \lambda (\theta )\Vert \left( v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta )g_i^\textsf {T}(\theta )v- g^*v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta )\right)\le & {} v^\textsf {T}\frac{1}{n}\sum _{i=1}^ng_i(\theta ). \end{aligned}$$

By C6 and C7, we have $\sup _{\Vert \theta -\theta ^*\Vert \le cn^{-1/2}}\Vert \frac{1}{n}\sum _{i=1}^ng_i(\theta )g_i^\textsf {T}(\theta )\Vert =O_p(1)$. Moreover, by using the Markov’s inequality, we have $g^*=o_p(n^{1/2})$. From these facts and (A.13), we obtain $\lambda (\theta )=O_p(n^{-1/2})$, uniformly for any $\theta \in B_0$. Now, write

$$\begin{aligned} 0= & {} \frac{1}{n}\sum _{i=1}^ng_i(\theta )/(1+U_i)\\= & {} \frac{1}{n}\sum _{i=1}^ng_i(\theta )\left( 1-U_i+\frac{U_i^2}{1+U_i}\right) \\= & {} \frac{1}{n}\sum _{i=1}^ng_i(\theta )-\frac{1}{n}\sum _{i=1}^ng_i(\theta )g_i^\textsf {T}(\theta )\lambda +\frac{1}{n}\sum _{i=1}^ng_i(\theta )\frac{U_i^2}{1+U_i}. \end{aligned}$$

Because

$$\begin{aligned} \left\| \frac{1}{n}\sum _{i=1}^ng_i(\theta )\frac{U_i^2}{1+U_i}\right\|\le & {} \frac{1}{n}\sum _{i=1}^n\left\| g_i(\theta )\right\| ^3 \Vert \lambda \Vert ^2|1+U_i|^{-1}\\= & {} o_p(n^{1/2})O_p(n^{-1})O_p(1)=o_p(n^{-1/2}), \end{aligned}$$

we have, uniformly for $\theta \in B_0$,

$$\begin{aligned} \lambda (\theta )= & {} \left\{ \frac{1}{n}\sum _{i=1}^n g_i(\theta )g_i^\textsf {T}(\theta )\right\} ^{-1} \frac{1}{n}\sum _{i=1}^n g_i(\theta )+o_p(n^{-1/2}).\ \ \end{aligned}$$

$\square$

Proof of Theorem 3

It follows from directional differentiation that minimizing

$$\begin{aligned} L_{2n}(\beta )=\sum _{i=1}^n \sum _{j=1}^n \frac{\hat{p}_i \hat{p}_j\delta _i\delta _j}{\pi (y_i,z_i,\hat{\gamma })\pi (y_j,z_j,\hat{\gamma })} \left| e_i(\beta )-e_j(\beta ) \right| \end{aligned}$$

is equivalent to solving $U_{2n}(\beta )=0$, where

$$\begin{aligned} U_{2n}(\beta )= & {} \sum _{i=1}^n\sum _{j=1}^n \frac{\hat{p}_i\hat{p}_j\delta _i\delta _j(w_i-w_j)}{\pi (y_i,z_i,\hat{\gamma })\pi (y_j,z_j,\hat{\gamma })} \text{ sgn }(e_i(\beta )-e_j(\beta )). \end{aligned}$$

From the asymptotic linearity of $U_n^*$ in (A.9), we have

$$\begin{aligned} n^{1/2}(\hat{\beta }_{ELW}-\beta ^*)=A_n^{-1}\{n^{1/2}U_n^*(\hat{\beta }_{ELW})-n^{1/2}U_n^*(\beta ^*)\}+o_p(1). \end{aligned}$$

(A.14)

For fixed estimators $\hat{\alpha }$, $\hat{\beta }_{IPW}$ and $\hat{\gamma }$, the Lagrange multiplier $\hat{\lambda }$ satisfies the constraint equations $G(\hat{\lambda },\hat{\alpha },\hat{\beta }_{IPW},\hat{\gamma })=0$. By Lemma A.3, it follows that $\hat{\lambda }$ is $n^{1/2}$-consistent for 0 as $n\rightarrow \infty$. Using Lemma A.3, C7 and the fact that $(\hat{\lambda },\hat{\alpha },\hat{\beta }_{IPW},\hat{\gamma })$ is $n^{1/2}$-consistent for $(0,\alpha ^*,\beta ^*,\gamma ^*)$, we have

$$\begin{aligned} \hat{\lambda }=S_g^{-1}U_g+S_g^{-1}G_\gamma (\hat{\gamma }-\gamma ^*)+o_p(n^{-1/2}). \end{aligned}$$

(A.15)

By (A.15), we have

$$\begin{aligned}&0\doteq U_{2n}(\hat{\beta }_{ELW})\\&\quad =U_n^*(\hat{\beta }_{ELW})+F_\gamma (\hat{\gamma }-\gamma )-F_g\hat{\lambda }+o_p(n^{-1/2})\\&\quad =U_n^*(\hat{\beta }_{ELW})-F_g S_g^{-1}U_g+(F_\gamma -F_g S_g^{-1}G_\gamma )S_B^{-1}U_B+o_p(n^{-1/2}), \end{aligned}$$

where

$$\begin{aligned} F_g= & {} \lim _{n\rightarrow \infty }n^{-2}\sum _{i=1}^n\sum _{j=1}^n \frac{\delta _i\delta _j(w_i-w_j)(g_i+g_j)^\textsf {T}}{\pi (y_i,z_i,\gamma ^*)\pi (y_j,z_j,\gamma ^*)}\text{ sgn }(e_i-e_j)\\= & {} \lim _{n\rightarrow \infty }2n^{-2}\sum _{i=1}^n\sum _{j=1}^n \frac{\delta _i\delta _j(w_i-w_j)g_i^\textsf {T}}{\pi (y_i,z_i,\gamma ^*)\pi (y_j,z_j,\gamma ^*)}\text{ sgn }(e_i-e_j). \end{aligned}$$

It is easy to verify that $F_\gamma =F_g S_g^{-1}G_\gamma$. Thus, $U_n^*(\hat{\beta }_{ELW})=F_g S_g^{-1}U_g+o_p(n^{-1/2})$. Combining this fact and (A.14), we have

$$\begin{aligned} n^{1/2}(\hat{\beta }_{ELW}-\beta ^*)=-A_n^{-1}\{n^{1/2}U_n^*(\beta ^*)-n^{1/2}F_g S_g^{-1}U_g\}+o_p(1). \end{aligned}$$

A little algebra reveals that

$$\begin{aligned} -F_gS_g^{-1}U_g= & {} -(F_h-F_\gamma S_B^{-1}H_{\gamma }^\textsf {T})(S_h-H_{\gamma }S_B^{-1}H_{\gamma }^\textsf {T})^{-1}U_h\nonumber \\&+\, \{F_\gamma -(F_h-F_\gamma S_B^{-1}H_{\gamma }^\textsf {T})(S_h-H_{\gamma }S_B^{-1}H_{\gamma }^\textsf {T})^{-1}H_{\gamma }\}S_B^{-1}U_{B}. \end{aligned}$$

Using this equation, we get that

$$\begin{aligned}&\lim _{n\rightarrow \infty }\text{ var }(n^{1/2}U_n^*(\beta ^*)-n^{1/2}F_g S_g^{-1}U_g)\\&\quad =S_\varphi -F_gS_g^{-1}F_g^\textsf {T}\\&\quad =S_\varphi -F_\gamma S_B^{-1}F_\gamma ^\textsf {T}-(F_\gamma S_B^{-1}H_{\gamma }^\textsf {T}-F_h)(S_h-H_{\gamma }S_B^{-1}H_{\gamma }^\textsf {T})^{-1}(F_\gamma S_B^{-1}H_{\gamma }^\textsf {T}-F_h)^\textsf {T}. \end{aligned}$$

Then, we have $n^{1/2}(\hat{\beta }_{ELW}-\beta ^*)$ converges to $N(0,\varSigma _{ELW})$. $\square$

Lemma A.4

Let $A(\beta ,\varSigma )=2n^{-2}\sum _{i\ne j}\frac{\delta _i\delta _j{\varPsi }_{ij}(\beta ,\varSigma )}{\pi (y_i,z_i,\hat{\gamma })\pi (y_j,z_j,\hat{\gamma })}(w_i-w_j)(w_i-w_j)^\textsf {T}$, where ${\varPsi }_{ij}(\beta ,\varSigma )=\frac{1}{\sigma _{ij}(\varSigma )}\phi \left( \frac{e_i(\beta )-e_j(\beta )}{\sigma _{ij}(\varSigma )}\right)$, $e_i(\beta )=y_i-w_{i}^\textsf {T}\beta$, $\sigma _{ij}^2(\varSigma )=(w_i-w_j)^\textsf {T}\varSigma (w_i-w_j)/n$, $\varSigma$ is some symmetric, positive definite matrix and $\phi$ denotes the standard normal density function. Then

$$\begin{aligned} \sup _{\Vert \beta -\beta ^*\Vert <cn^{-1/2}}\Vert {A}(\beta ,\varSigma )- A\Vert {\mathop {\longrightarrow }\limits ^{\textstyle p}}0,\\ \end{aligned}$$

where $A=\lim _{n\rightarrow \infty }2n^{-2}\sum _{i=1}^n\sum _{j=1}^n(w_i-w_j)(w_i-w_j)^\textsf {T}\int _{-\infty }^{\infty }f_i(u)d F_j(u)$.

Proof of Lemma A.4

Let $A_n=2n^{-2}\sum _{i=1}^n\sum _{j=1}^n(w_i-w_j)(w_i-w_j)^\textsf {T}\int _{-\infty }^{\infty }f_i(u)d F_j(u)$. By the triangle inequality, we have

$$\begin{aligned}&\sup _{\Vert \beta -\beta ^*\Vert<cn^{-1/2}}\Vert {A}(\beta ,\varSigma )- A\Vert \\&\quad \le \sup _{\Vert \beta -\beta ^*\Vert <cn^{-1/2}}\Vert {A}(\beta ,\varSigma )- {A}(\beta ,\varSigma ^*)\Vert \\&\qquad +\Vert {A}(\beta ,\varSigma ^*)-E\{{A}(\beta ,\varSigma ^*)\}\Vert +\Vert E\{{A}(\beta ,\varSigma ^*)\}-A_n\Vert +\Vert A_n-A\Vert . \end{aligned}$$

It is easy to see that $\Vert A_n-A\Vert {\mathop {\longrightarrow }\limits ^{\textstyle p}}0$ and $\lim _{n\rightarrow \infty }\Vert {A}(\beta ,\varSigma ^*)-E\{{A}(\beta ,\varSigma ^*)\}\Vert {\mathop {\longrightarrow }\limits ^{\textstyle p}}0$. Observe that

$$\begin{aligned}&\sup _{\Vert \beta -\beta ^*\Vert<cn^{-1/2}}\Vert {A}(\beta ,\varSigma )- {A}(\beta ,\varSigma ^*)\Vert \\&\quad \le \sup _{\Vert \beta -\beta ^*\Vert <cn^{-1/2}}2n^{-2}c_0^{-2}\sum _{i\ne j} \Vert w_i-w_j\Vert ^2 |{\varPsi }_{ij}(\beta ,\varSigma )-{\varPsi }_{ij}(\beta ,\varSigma ^*)|\\&\quad \le 2c_0^{-2} cn^{-1/2}n^{-2}\sum _{i\ne j} \Vert w_i-w_j\Vert ^3 \sigma _{ij}^{-2}(\varSigma ) \left| \phi '\left( \frac{e_i(\bar{\beta })-e_j(\bar{\beta })}{\sigma _{ij}(\varSigma )}\right) \right| , \end{aligned}$$

where $\phi '(u)$ is the derivative of u and $\bar{\beta }$ is a point on the segment connecting $\beta$ and $\beta ^*$. Note that $\sigma _{ij}(\varSigma )=O_p(n^{-1/2})$ and $\lim _{u\rightarrow \infty }|u\phi '(u)|=0$, then we have $\sup _{\Vert \beta -\beta ^*\Vert <cn^{-1/2}}\Vert {A}(\beta ,\varSigma )- {A}(\beta ,\varSigma ^*)\Vert {\mathop {\longrightarrow }\limits ^{\textstyle p}}0$. Next, we will show that $\Vert E\{{A}(\beta ,\varSigma ^*)\}-A_n\Vert {\mathop {\longrightarrow }\limits ^{\textstyle p}}0$. Notice that

$$\begin{aligned}&\Vert E\{{A}(\beta ,\varSigma ^*)\}-A_n\Vert \\&\quad \le 2n^{-2}\sum _{i\ne j} \Vert w_i-w_j\Vert ^2\left| \frac{1}{\sigma _{ij}(\varSigma )}\int _{-\infty }^{\infty } \phi \left( \frac{u}{\sigma _{ij}(\varSigma )}\right) f_{ij}(u)du -\int _{-\infty }^{\infty }f_i(u)d F_j(u) \right| . \end{aligned}$$

Since

$$\begin{aligned} f_{ij}(u)= & {} \int _{-\infty }^{\infty } f_i(u+v)f_j(v)dv, \end{aligned}$$

we have

$$\begin{aligned}&\left| \frac{1}{\sigma _{ij}(\varSigma )}\int _{-\infty }^{\infty } \phi \left( \frac{u}{\sigma _{ij}(\varSigma )}\right) f_{ij}(u)du -\int _{-\infty }^{\infty }f_i(u)d F_j(u) \right| \\&\quad = \left| \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\phi (u) f_i(u\sigma _{ij}(\varSigma )+v)f_j(v)dvdu -\int _{-\infty }^{\infty }f_i(u)f_j(u)du \right| \\&\quad = \left| \int _{-\infty }^{\infty }\int _{-\infty }^{\infty }\phi (u) [f_i(v)+f_i'(\xi _u) u\sigma _{ij}(\varSigma )]f_j(v)dvdu -\int _{-\infty }^{\infty }f_i(u)f_j(u)du \right| \\&\quad \le \sigma _{ij}(\varSigma )\left| \int _{-\infty }^{\infty }\int _{-\infty }^{\infty } u\phi (u) f_i'(\xi _u)f_j(v)dvdu\right| , \end{aligned}$$

where $\xi _u$ lies between 0 and $u\sigma _{ij}(\varSigma )$. By condition C4, there exists $M>0$, such that $\sup _{1\le i \le n}|f_{i}'(u)|<M$. It follows that

$$\begin{aligned} \Vert E\{{A}(\beta ,\varSigma ^*)\}-A_n\Vert \le 2n^{-2}\sum _{i\ne j} \Vert w_i-w_j\Vert ^2\sigma _{ij}(\varSigma ) \sqrt{\frac{2}{\pi }}M\rightarrow 0. \end{aligned}$$

The desired result follows. $\square$