Statistical Inference in Marginalized Zero-inflated Poisson Regression Models with Missing Data in Covariates

Abstract

The marginalized zero-inflated poisson (MZIP) regression model quantifies the effects of an explanatory variable in the mixture population. Also, in practice the variables are usually partially observed. Thus, we first propose to study the maximum likelihood estimator when all variables are observed. Then, assuming that the probability of selection is modeled using mixed covariates (continuous, discrete and categorical), we propose a semiparametric inverse-probability weighted (SIPW) method for estimating the parameters of the MZIP model with covariates missing at random (MAR). The asymptotic properties (consistency, asymptotic normality) of the proposed estimators are established under certain regularity conditions. Through numerical studies, the performance of the proposed estimators was evaluated. Then the results of the SIPW are compared to the results obtained by semiparametric inverse-probability weighted kermel-based (SIPWK) estimator method. Finally, we apply our methodology to a dataset on health care demand in the United States.

Appendix A

PROOFS OF ASYMPTOTIC RESULTS

6.1 Proof of Theorem 1

We prove consistency of $\hat{\theta}_{n}^{F}$ by checking the conditions of the inverse function theorem of Foutz [13]. These conditions are proved in a series of technical lemmas.

Lemma 1. As $n\rightarrow\infty$, $n^{-1/2}U_{F,n}(\theta_{0})$ converges in probability to 0.

Proof of Lemma 1. Decompose $n^{-1/2}U_{F,n}(\theta_{0})$ as, for every $i=1,\ldots,n$, we have

$$n^{-1/2}U_{F,n}(\theta_{0})=\begin{pmatrix}\displaystyle\frac{1}{n}\sum_{i=1}^{n}Z_{i1}B_{i}(\theta_{0})\\ \vdots\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}Z_{iq}B_{i}(\theta_{0})\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}X_{i1}A_{i}(\theta_{0})\\ \vdots\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}X_{ip}A_{i}(\theta_{0})\end{pmatrix}.$$

For $i=1,\ldots,n$ and $l=1,\ldots,q$;

$$\mathbb{E}\left[Z_{il}B_{i}(\theta_{0})\right]=\mathbb{E}\left[\mathbb{E}\left[Z_{il}B_{i}(\theta_{0})|\mathbf{X}_{i},\mathbf{Z}_{i}\right]\right]=\mathbb{E}\left[Z_{il}\mathbb{E}\left[B_{i}(\theta_{0})|\mathbf{X}_{i},\mathbf{Z}_{i}\right]\right].$$

We have

$$\mathbb{E}[B_{i}(\theta_{0})|\mathbf{X}_{i},\mathbf{Z}_{i}]=\frac{\mathbb{E}(J_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})e^{\mathbf{Z}^{T}_{i}\gamma_{0}}\left(e^{h_{i}(\theta_{0})}-e^{\mathbf{X}^{T}_{i}\alpha_{0}}\right)}{e^{\mathbf{Z}^{T}_{i}\gamma_{0}+h_{i}(\theta_{0})}+1}+\frac{e^{\mathbf{Z}^{T}_{i}\gamma_{0}}(\mathbb{E}(Y_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})-1)}{1+e^{\mathbf{Z}^{T}_{i}\gamma_{0}}}$$

$${}-\left[1-\mathbb{E}(J_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})\right]e^{\mathbf{X}^{T}_{i}\alpha_{0}+\mathbf{Z}^{T}_{i}\gamma_{0}}.$$

Now, we have

$\mathbb{E}(J_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})=\mathbb{P}(Y_{i}=0|\mathbf{X}_{i},\mathbf{Z}_{i}),$ $\mathbb{E}(Y_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})=\nu_{i}$ and $\mathbb{E}(1-J_{i}|\mathbf{X}_{i},\mathbf{Z}_{i})=\mathbb{P}(Y_{i}>0|\mathbf{X}_{i},\mathbf{Z}_{i}).$ It follows that $\mathbb{E}[Z_{il}B_{i}(\theta_{0})]=0$.

Using similarly arguments we prove that, for every $i=1,\ldots,n$ and $j=1,\ldots,p$, $\mathbb{E}[X_{ij}A_{i}(\theta_{0})]=0$.

Now, for every $i=1,\ldots,n$ and $l=1,\ldots,q$, we have

$$\textrm{var}\left(Z_{il}B_{i}(\theta_{0})\right)\leq\mathbb{E}\left(Z_{il}^{2}B_{i}^{2}(\theta_{0})\right).$$

By $\mathbf{H3}$, we have $\mathbb{E}\left(Z_{il}^{2}B_{i}^{2}(\theta_{0})\right)<\infty$.

Using similar arguments, we prove $\textrm{var}\left(X_{ij}A_{i}(\theta_{0})\right)<\infty$ for every $i=1,\ldots,n$ and $j=1,\ldots,p$.

Thus, by the weak law of large numbers, $n^{-1/2}U_{F,n}(\theta_{0})$ converges in probability to $0$, which concludes the proof.

Lemma 2. As $n\rightarrow\infty$, $n^{-1/2}\frac{\partial U_{F,n}(\theta)}{\partial\theta^{T}}$ converges in probability to a fixed function $-\Sigma(\theta)$, uniformly in an open neighbourhood of $\theta_{0}$.

Proof of Lemma 2: Let $\tilde{U}_{F,n}(\theta):=n^{-1/2}\frac{\partial U_{F,n}(\theta)}{\partial\theta^{T}}$, and $\nu_{\theta_{0}}$ be an open neighbourhood of $\theta_{0}$. Let $\theta\in\nu_{\theta_{0}}$.

By the weak law of large numbers and $\mathbf{H3}$, $\tilde{U}_{F,n}(\theta)=\frac{1}{n}\sum_{i=1}^{n}\left\{\frac{\partial^{2}l_{i}(\theta)}{\partial\theta\partial\theta^{T}}\right\}$ converges in probability to the matrix $-\Sigma(\theta)$ as $n\rightarrow\infty$, where $\Sigma(\theta)=\mathbb{E}\left[-\frac{\partial^{2}l_{1}(\theta)}{\partial\theta\partial\theta^{T}}\right]$.

By conditions $\mathbf{H4}$, we prove that the convergence of $\tilde{U}_{F,n}(\theta)$ to $-\Sigma(\theta)$ is uniform on $\nu_{\theta_{0}}$.

The conditions inverse function theorem of Foutz [13] are verified. Finally $\hat{\theta}_{n}$ converges in probability to $\theta_{0}$.

Now, we prove that $\hat{\theta}_{n}^{F}$ is asymptotically Gaussian. To do this, it follows by a Taylor’s expansion of $U_{F,n}(\hat{\theta}_{F,n})$ at $\theta_{0}$ yields

$$0=U_{F,n}(\theta_{0})+\tilde{U}_{F,n}(\theta_{0})\sqrt{n}(\hat{\theta}_{n}^{F}-\theta_{0})+o_{p}(1)$$

.

By calculations $\textrm{var}(U_{F,n}(\theta_{0}))=\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}\left(\dot{l_{i}}(\theta_{0})\dot{l_{i}}(\theta_{0})^{T}\right)=Q_{F}(\theta_{0})$.

Finally, by Lemma 2 and Slusky’s theorem, $\sqrt{n}(\hat{\theta}_{n}^{F}-\theta_{0})$ converges in distribution to the Gaussian vector of mean zero and variance $\Delta_{F}$, where $\Delta_{F}$ is defined in Theorem 1.

Appendix B

6.2 Proof of Theorem 2

We prove consistency of $\hat{\theta}_{n}^{ws}$ by checking the conditions of the inverse function theorem of Foutz [13]. These conditions are proved in a series of technical lemmas.

Lemma 3. As $n\rightarrow\infty$, $n^{-1/2}U_{w,n}(\theta_{0},\hat{\pi})$ converges in probability to $0$.

Proof of Lemma 3. We decompose $n^{-1/2}U_{w,n}(\theta_{0},\hat{\pi})$ as

$$n^{-1/2}U_{ws,n}(\theta_{0},\hat{\pi})=(n^{-1/2}U_{ws,n}(\theta_{0},\hat{\pi})-n^{-1/2}U_{ws,n}(\theta_{0},\pi))+n^{-1/2}U_{ws,n}(\theta_{0},\pi).$$

(6.1)

Considering the first term of this decomposition.

Let $\mathbf{S}^{\prime}_{i}=(\mathbf{S}^{D}_{i},\mathbf{S}^{\prime,D}_{i})$ and $G_{n}(\theta_{0},\pi)=n^{-1/2}U_{ws,n}(\theta_{0},\hat{\pi})-n^{-1/2}U_{ws,n}(\theta_{0},\pi)$, we have

$$G_{n}(\theta_{0},\pi)=\frac{1}{n}\sum_{i=1}^{n}\Delta_{i}\left(\frac{1}{\hat{\pi}(Y_{i},\mathbf{S}^{\prime}_{i})}-\frac{1}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right)\dot{l_{i}}(\theta_{0}),$$

$${}=\frac{1}{n}\sum_{i=1}^{n}\Delta_{i}\left[\frac{\hat{\pi}(Y_{i},\mathbf{S}^{\prime}_{i})-\pi(Y_{i},\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}+O_{P}\left((\hat{\pi}(Y_{i},\mathbf{S}^{\prime}_{i})-\pi(Y_{i},\mathbf{S}^{\prime}_{i}))^{2}\right)\right]\dot{l_{i}}(\theta_{0}),$$

$${}=\frac{1}{n}\sum_{i=1}^{n}\Delta_{i}\left[\frac{\frac{\sum_{k=1}^{n}\Delta_{k}I(Y_{k}=y,\mathbf{S}^{\prime}_{k}=s^{\prime})}{\sum_{i=1}^{n}I(Y_{i}=y,\mathbf{S}^{\prime}_{i}=s^{\prime})}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})+o^{*}_{p}\left(\frac{1}{\sqrt{n}}\right),$$

$${}=\frac{1}{n}\sum_{i=1}^{n}\Delta_{i}\left[\frac{\frac{1}{n}\sum_{i=1}^{n}\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i}\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}+O_{p}\left(\frac{1}{n}\right)\right]$$

$${}\times\dot{l_{i}}(\theta_{0})+o^{*}_{p}\left(\frac{1}{\sqrt{n}}\right),$$

$${}=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{{}^{\prime}}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})$$

$${}+\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})+o_{p}^{*}\left(\frac{1}{\sqrt{n}}\right),$$

were $o_{p}^{*}(a_{n})$ denotes a matrix whose components are uniformly $o_{p}(a_{n})$. By the weak law of large numbers we have

$$\frac{1}{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]$$

converges in probability to $0$ as $n\rightarrow\infty$.

Using conditions $\mathbf{H3}$, we prove that $\dot{l_{i}}(\theta_{0})$ is finite a.s. Finally, by Slutsky’s theorem

$$G_{n}(\theta_{0},\pi)=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi_{i}^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]$$

$${}\times\dot{l_{i}}(\theta_{0})+\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})$$

converges in probability to $0$ as $n\rightarrow\infty$.

Next, consider the term $n^{-1/2}U_{ws,n}(\theta_{0},\pi(Y_{i},\mathbf{S}^{\prime}_{i}))$ in decomposition (6.1).

We show that $n^{-1/2}U_{ws,n}(\theta_{0},\pi(Y_{i},\mathbf{S}^{\prime}_{i}))$ converges in probability to $0$ as $n\rightarrow\infty$.

For every $i=1,\ldots,n$, we have

$$n^{-1/2}U_{ws,n}(\theta_{0},\pi(Y_{i},\mathbf{S}^{\prime}_{i}))=\begin{pmatrix}\displaystyle\frac{1}{n}\sum_{i=1}^{n}\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{i1}B_{i}(\theta_{0})\\ \vdots\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{iq}B_{i}(\theta_{0})\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}X_{i1}A_{i}(\theta_{0})\\ \vdots\\ \displaystyle\frac{1}{n}\sum_{i=1}^{n}\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}X_{ip}A_{i}(\theta_{0})\end{pmatrix}.$$

For $i=1,\ldots,n$ and $l=1,\ldots,q$;

$$\mathbb{E}\left[\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}B_{i}(\theta_{0})\right]=\mathbb{E}\left[\mathbb{E}\left[\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{i1}B_{i}(\theta_{0})|Y_{i},\mathbf{S}^{\prime}_{i}\right]\right].$$

Two cases should be considered, namely: (i) $Z_{il}$ is a component of $Z^{\textrm{obs}}$ and (ii) $Z_{il}$ is a component of $Z^{\textrm{miss}}$. In case (i), we have

$$\mathbb{E}\left[\mathbb{E}\left[\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}B_{i}(\theta_{0})|Y_{i},\mathbf{S}^{\prime}_{i}\right]\right]=\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}\mathbb{E}[\Delta_{i}B_{i}(\theta_{0})|Y_{i},\mathbf{S}^{\prime}_{i}]\right].$$

Given $\mathbf{V}_{i}=(Y_{i},\mathbf{S}^{\prime}_{i})$, $Z_{il}B_{i}(\theta_{0})$ is a function of $(\mathbf{X}^{\textrm{miss}},\mathbf{Z}^{\textrm{miss}})$ only. Thus, by the MAR assumption, $B_{i}(\theta_{0})$ and $\Delta_{i}$ are independent

$$\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}\mathbb{E}[\Delta_{i}B_{i}(\theta_{0})|\mathbf{V}_{i}]\right]=\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}\mathbb{E}[\Delta_{i}|\mathbf{V}_{i}]\mathbb{E}[B_{i}(\theta_{0})|\mathbf{V}_{i}]\right],$$

$${}=\mathbb{E}\left[Z_{il}\mathbb{E}[B_{i}(\theta_{0})|\mathbf{V}_{i}]\right],$$

$${}=\mathbb{E}\left[Z_{il}B_{i}(\theta_{0})\right],$$

$${}=0.$$

In case (ii),

$$\mathbb{E}\left[\mathbb{E}\left[\frac{\Delta_{i}}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{i1}B_{i}(\theta_{0})|\mathbf{V}_{i}\right]\right]=\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}\mathbb{E}[\Delta_{i}Z_{il}B_{i}(\theta_{0})|\mathbf{V}_{i}]\right].$$

Given $\mathbf{V}_{i}$, $Z_{il}B_{i}(\theta_{0})$ is a function of $(\mathbf{X}^{\textrm{miss}},\mathbf{Z}^{\textrm{miss}})$ only. Thus, by the MAR assumption, $B_{i}(\theta_{0})$ and $\Delta_{i}$ are independent

$$\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}\mathbb{E}[\Delta_{i}Z_{il}B_{i}(\theta_{0})|\mathbf{V}_{i}]\right]=\mathbb{E}\left[\frac{1}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}\mathbb{E}[\Delta_{i}|V_{i}]\mathbb{E}[Z_{il}B_{i}(\theta_{0})|\mathbf{V}_{i}]\right],$$

$${}=\mathbb{E}[Z_{il}B_{i}(\theta_{0})],$$

$${}=0.$$

It follows that $\mathbb{E}[\frac{\Delta_{i}}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}B_{i}(\theta_{0})]=0$.

Using similar arguments, we prove that $\mathbb{E}[\frac{\Delta_{i}}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}X_{ij}A_{i}(\theta_{0})]=0$.

Now, for every $i=1,\ldots,n$ and $l=1,\ldots,q$ , we have

$$\textrm{var}\left(\frac{\Delta_{i}}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}B_{i}(\theta_{0})\right)\leq\mathbb{E}\left(\frac{\Delta_{i}}{\pi_{i}^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}^{2}B_{i}^{2}(\theta_{0})\right).$$

By $\mathbf{H3}$, we have $\mathbb{E}\left(\frac{\Delta_{i}}{\pi_{i}^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}Z_{il}^{2}B_{i}^{2}(\theta_{0})\right)<\infty$ .

Using similar arguments, we prove

$$\textrm{var}\left(\frac{\Delta_{i}}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}X_{ij}A_{i}(\theta_{0})\right)<\infty \text{for every} i=1,\ldots,n \textrm{and} j=1,\ldots,p.$$

Thus, by the weak law of large numbers, $n^{-1/2}U_{ws,n}(\theta_{0},\pi(Y_{i},\mathbf{S}^{\prime}_{i}))$ converges in probability to $0$ as $n\rightarrow\infty$.

Finally $n^{-1/2}U_{w,n}(\theta_{0},\hat{\pi}(Y_{i},\mathbf{S}^{\prime}_{i}))$ converges to $0$, which concludes the proof.

Lemma 4. As $n\rightarrow\infty$, $n^{-1/2}\frac{\partial U_{ws,n}(\theta,\hat{\pi})}{\partial\theta^{T}}$ converges in probability to a fixed function $-\Sigma(\theta)$, uniformly in a neighbourhood of $\theta_{0}$.

Proof of Lemma 4. Let $\bar{U}_{ws,n}(\theta,\hat{\pi}):=n^{-1/2}\frac{\partial U_{ws,n}(\theta,\pi)}{\partial\theta^{T}}$ and $\ddot{l_{i}}(\theta)=\frac{\partial^{2}l_{i}(\theta)}{\partial\theta\partial\theta^{T}}$. We have

$$\bar{U}_{ws,n}(\theta,\hat{\pi})=\left[\bar{U}_{ws,n}(\theta,\hat{\pi})-\bar{U}_{ws,n}(\theta,\pi)\right]+\bar{U}_{ws,n}(\theta,\pi).$$

Using similary argument in Lemma 4, we have $\bar{U}_{ws,n}(\theta,\hat{\pi})-\bar{U}_{ws,n}(\theta,\pi)$ converges in probability to $0$. By the weak law of large numbers, and $\mathbf{H3}$

$$\bar{U}_{ws,n}(\theta,\pi)=\frac{1}{n}\sum_{i=1}^{n}\left\{\frac{\Delta_{i}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\ddot{l_{i}}(\theta)\right\}$$

converges in probability to the matrix $-\Sigma(\theta)$ as $n\rightarrow\infty$.

By $\mathbf{H5}$, we prove that the convergence of $\tilde{U}_{ws,n}(\theta,\hat{\pi})$ to $-\Sigma(\theta)$ is uniform.

The conditions inverse function theorem Foutz [13] are verified. Finally $\hat{\theta}_{n}^{ws}$ converges in probability to $\theta_{0}$.

Now, we prove that $\theta_{n}^{ws}$ is asymptotically Gaussian.

It follows by a Taylor’s expansion of $U_{ws,n}(\hat{\theta}^{ws}_{n},\hat{\pi})$ at $(\theta_{0},\hat{\pi})$ yields

$$0=U_{ws,n}(\hat{\theta}^{ws}_{n},\hat{\pi})=U_{ws,n}(\theta_{0},\hat{\pi})+\bar{U}_{ws,n}(\theta,\hat{\pi})\sqrt{n}(\hat{\theta}^{ws}_{n}-\theta_{0})+o_{p}(1),$$

therefore

$$\sqrt{n}(\hat{\theta}^{ws}_{n}-\theta_{0})=-\left[\bar{U}_{ws,n}(\theta_{0},\hat{\pi})\right]^{-1}U_{ws,n}(\theta_{0},\hat{\pi})+o_{p}(1),$$

thus

$$\sqrt{n}(\hat{\theta}^{ws}_{n}-\theta_{0})=\Sigma^{-1}(\theta_{0})U_{ws,n}(\theta_{0},\hat{\pi})+\left[-\bar{U}_{ws,n}^{-1}(\theta_{0},\hat{\pi})-\Sigma^{-1}(\theta_{0})\right]U_{ws,n}(\theta_{0},\hat{\pi})+o_{p}(1).$$

By calculations,

$$\textrm{Var}\left[U_{ws,n}(\theta_{0},\hat{\pi})\right]=\textrm{Var}\left\{U_{ws,n}(\theta_{0},\pi)+\left[U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)\right]\right\},$$

$${}=\textrm{Var}\left[U_{ws,n}(\theta_{0},\pi)\right]+\textrm{Var}\left[U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)\right]$$

$${}+2\textrm{Cov}\left[U_{ws,n}(\theta_{0},\pi),U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)\right],$$

$$\textrm{Var}\left[U_{ws,n}(\theta_{0},\pi)\right]=\mathbb{E}\left[\frac{\dot{l}_{i}(\theta_{0},\pi)\dot{l}_{i}(\theta_{0},\pi)^{T}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right]=\Omega_{3}(\theta_{0},\pi).$$

Let $H(\theta_{0},\pi)=U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)$

$$H(\theta_{0},\pi)=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\Delta_{i}\left(\frac{1}{\hat{\pi}(Y_{i},\mathbf{S}^{\prime}_{i})}-\frac{1}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right)\dot{l_{i}}(\theta_{0}),$$

$${}=-\frac{1}{\sqrt{n^{3}}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})$$

$${}-\frac{1}{\sqrt{n^{3}}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})+o_{p}(1),$$

$${}=-\mathbf{Q}_{1n}-\mathbf{Q}_{2n}+o_{p}(1),$$

where $o_{p}(a_{n})$ denotes a column vector whose components are uniformly $o_{p}(a_{n})$.

$$\mathbf{Q}_{1n}=\frac{1}{\sqrt{n^{3}}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0}).$$

$$\mathbf{Q}_{2n}=\frac{1}{\sqrt{n^{3}}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0}).$$

Let

$$P_{ik}=\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0}).$$

In order to show that $\mathbf{Q}_{1n}=\mathbf{O}_{p}(1/\sqrt{n}),$ $\mathbb{E}(\mathbf{Q}_{1n})=\mathbf{O}_{p}(1/\sqrt{n})$ and $\textrm{Var}(\mathbf{Q}_{1n})=\mathbf{O}_{p}^{*}(1/n)$ where $O^{*}(a_{n})$ and $O(a_{n})$ denote a matrix and column vector whose components are uniformly $O(a_{n})$. It first can be shown that

$$\mathbb{E}\left[P_{ik}\right]=\mathbb{E}\left[\mathbb{E}(P_{ik}|Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})\right]=\begin{cases}0\quad\text{if}\quad i\neq k\\ \mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\dot{l_{i}}(\theta_{0})}{\pi_{i}(Y_{i},\mathbf{S}^{\prime}_{i})}\right]\quad\text{if}\quad i=k,\end{cases}$$

and then

$$\mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}_{i})\right]\dot{l_{i}}(\theta_{0})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right]=\mathbb{E}\left\{\mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\dot{l_{i}}(\theta_{0})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}|Y_{i},\mathbf{S}^{\prime}_{i}\right]\right\},$$

$${}=\mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\dot{l}_{i}^{*}(\theta_{0})\right].$$

Thus, we have

$$\mathbb{E}(\mathbf{Q}_{1n})=\frac{1}{n^{3/2}}\sum_{k=1}^{n}\sum_{i=1}^{n}\mathbb{E}\left[P_{ik}\right]=\frac{1}{n^{3/2}}\sum_{i=1}^{n}\mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\dot{l}_{i}^{*}(\theta_{0})\right]=O(\frac{1}{\sqrt{n}}).$$

We have

$$\textrm{Cov}(P_{ij},P_{kl})=\mathbb{E}\bigg{\{}\left[\frac{\left[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]\left[\Delta_{j}-\pi(Y_{j},\mathbf{S}^{\prime}_{j})\right]I(Y_{j}=Y_{i},\mathbf{S}^{\prime}_{j}=\mathbf{S}^{\prime}_{i})}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0})$$

$${}\times\left[\frac{\left[\Delta_{k}-\pi(Y_{k},\mathbf{S}^{\prime}_{k})\right]\left[\Delta_{l}-\pi(Y_{l},\mathbf{S}^{\prime}_{l})\right]I(Y_{l}=Y_{k},\mathbf{S}^{\prime}_{l}=\mathbf{S}^{\prime}_{k})}{\pi^{2}(Y_{k},\mathbf{S}^{\prime}_{k})P(Y=Y_{k},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{k})}\right]\dot{l_{k}}(\theta_{0})^{T}\bigg{\}},$$

$$\textrm{Cov}(P_{ij},P_{lk})=\begin{cases}0\quad\text{if}\quad k\neq i,j\quad\text{and}\quad l\neq i,j\\ 0\quad\text{if}\quad k\neq i,j\quad\text{and}\quad l=i\quad\text{or}\quad j\\ \mathbb{E}\left[\frac{\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]^{2}\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}\right]\quad\text{if}\quad i=l\quad\text{and}\quad j=k\end{cases}$$

and

$$\textrm{Var}(\mathbf{Q}_{1n})=\frac{1}{n^{3}}\bigg{\{}\sum_{i,j}^{n}\textrm{Var}(P_{ij})+\sum_{i,j}^{n}\bigg{[}\sum_{l=j,k\neq j}\textrm{Cov}(P_{ij},P_{lk})$$

$${}+\sum_{k=j,l\neq j}\textrm{Cov}(P_{ij},P_{lk})+\sum_{k\neq j,l\neq j}\textrm{Cov}(P_{ij},P_{lk})\bigg{]}\bigg{\}},$$

$${}=\frac{1}{n^{3}}\bigg{\{}n\mathbb{E}\bigg{\{}\frac{\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}}{\pi^{4}(Y_{i},\mathbf{S}^{\prime}_{i})}\bigg{[}\pi(Y_{i},\mathbf{S}^{\prime}_{i})-4\pi^{4}(Y_{i},\mathbf{S}^{\prime}_{i})$$

$${}+6\pi^{3}(Y_{i},\mathbf{S}^{\prime}_{i})-3\pi^{4}(Y_{i},\mathbf{S}^{\prime}_{i})\bigg{]}\bigg{\}}$$

$${}+n(n-1)\mathbb{E}\left\{\frac{\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}}{\pi^{4}(Y_{i},\mathbf{S}^{\prime}_{i})}\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]^{2}\right\}$$

$${}+n(n-1)\mathbb{E}\left\{\frac{\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}}{\pi^{4}(Y_{i},\mathbf{S}^{\prime}_{i})}\left[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]^{2}\right\}\bigg{\}},$$

$${}=O^{*}(\frac{1}{n^{2}})+O^{*}(\frac{1}{n})+O^{*}(\frac{1}{n})$$

$${}=O^{*}(\frac{1}{n}).$$

Therefore, $\mathbf{Q}_{1n}=O_{p}(\frac{1}{\sqrt{n}})$, $\mathbf{Q}_{2n}$ can be expressed as follows:

$$\mathbf{Q}_{2n}=\frac{1}{\sqrt{n^{3}}}\sum_{i=1}^{n}\sum_{k=1}^{n}\left[\frac{\left[\Delta_{k}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})\right]I(Y_{k}=Y_{i},\mathbf{S}^{\prime}_{k}=\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})P(Y=Y_{i},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{i})}\right]\dot{l_{i}}(\theta_{0}),$$

$${}=\frac{1}{\sqrt{n}}\sum_{k=1}^{n}\left[\frac{\Delta_{k}-\pi(Y_{k},\mathbf{S}^{\prime}_{k})}{\pi(Y_{k},\mathbf{S}^{\prime}_{k})}\right]\left[\dot{l}_{k}^{*}(\theta_{0})+\frac{\frac{1}{n}\sum_{i=1}^{n}I(Y_{i}=Y_{k},\mathbf{S}^{\prime}_{i}=\mathbf{S}^{\prime}_{k})\dot{l_{i}}(\theta_{0})}{P(Y=Y_{k},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{k})}-\dot{l}_{k}^{*}(\theta_{0})\right],$$

$${}=\frac{1}{\sqrt{n}}\sum_{k=1}^{n}\left[\frac{\Delta_{k}-\pi(Y_{k},\mathbf{S}^{\prime}_{k})}{\pi(Y_{k},\mathbf{S}^{\prime}_{k})}\right]\dot{l}_{k}^{*}(\theta_{0})+\frac{1}{\sqrt{n}}\sum_{k=1}^{n}\Phi_{k}\left[\frac{1}{n}\sum_{i=1}^{n}\Psi_{ik}(\theta_{0})\right],$$

where $\Phi_{k}=\frac{\Delta_{k}-\pi(Y_{k},\mathbf{S}^{\prime}_{k})}{\pi(Y_{k},\mathbf{S}^{\prime}_{k})}$ and

$$\Psi_{ik}(\theta_{0})=\frac{I(Y_{i}=Y_{k},\mathbf{S}^{\prime}_{i}=\mathbf{S}^{\prime}_{k})\dot{l}_{i}(\theta_{0})-P(Y=Y_{k},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{k})\dot{l}_{k}^{*}(\theta_{0})}{P(Y=Y_{k},\mathbf{S}^{\prime}=\mathbf{S}^{\prime}_{k})}.$$

We have $\mathbb{E}\left[\Psi_{ik}(\theta_{0})|Y_{i}=Y_{k},\mathbf{S}^{\prime}_{i}=\mathbf{S}^{\prime}_{k}\right]=0$ and, hence,

$$\mathbb{E}\left[\Phi_{k}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\Psi_{ik}(\theta_{0})\right)\right]=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\mathbb{E}\left[\mathbb{E}\left(\Phi_{k}\Psi_{k}(\theta_{0})|Y_{i}=Y_{k},\mathbf{S}^{\prime}_{i}=\mathbf{S}^{\prime}_{k}\right)\right],$$

$$=0.$$

Let $\Psi_{iks}(\theta_{0})$ be the $s$th element of $\Psi_{ik}(\theta_{0})$. Then, by Cauchy–Schwarz’s inequality,

$$\mathbb{E}\left[\mid\Phi_{k}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\Psi_{iks}(\theta_{0})\right)\mid\right]\leq\left[\mathbb{E}\left(\Phi_{k}^{2}\right)\right]^{1/2}\left\{\mathbb{E}\left[\frac{1}{n}\left(\sum_{i=1}^{n}\Psi_{iks}(\theta_{0})\right)^{2}\right]\right\}^{1/2}.$$

Because for each element of $\Psi_{ik}(\theta_{0})$

$$\mathbb{E}\left[\frac{1}{n}\left(\sum_{i=1}^{n}\Psi_{iks}(\theta_{0})\right)^{2}\right]=\frac{1}{n}\left[\mathbb{E}\left(\sum_{i=1}^{n}\Psi_{iks}^{2}(\theta_{0})\right)+\sum_{i=1}^{n}\sum_{j=1,j\neq i}^{n}\mathbb{E}(\Psi_{iks}(\theta_{0})\Psi_{jks}(\theta_{0}))\right],$$

$${}=\mathbb{E}(\Psi_{iks}^{2}(\theta_{0}))<\infty,$$

we can proove $\mathbb{E}\left[\mid\Phi_{k}\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\Psi_{ik}(\theta_{0})\right)\mid\right]<\infty$.

By the weak law of large numbers $\frac{1}{n}\sum_{k=1}^{n}\left\{\Phi_{k}\left[\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\Psi_{ik}(\theta_{0})\right]\right\}=o_{p}(1)$ . Hence, $\mathbf{Q}_{2n}$ can be expressed as $\mathbf{Q}_{2n}=\frac{1}{\sqrt{n}}\sum_{k=1}^{n}\left[\frac{\Delta_{k}-\pi(Y_{k},\mathbf{S}^{\prime}_{k})}{\pi(Y_{k},\mathbf{S}^{\prime}_{k})}\right]\dot{l}_{k}^{*}(\theta_{0}+o_{p}(1)$.

$$\textrm{Var}\left[U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)\right]=\mathbb{E}\left\{\mathbb{E}\left[\frac{[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})]^{2}\left[\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}|Y_{i},\mathbf{S}^{\prime}_{i}\right]\right\}$$

$${}+o^{*}(1),$$

$${}=\mathbb{E}\left[\frac{[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})]\left[\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right]+o^{*}(1)$$

and let $\Sigma=\textrm{Cov}\left[U_{ws,n}(\theta_{0},\pi),U_{ws,n}(\theta_{0},\hat{\pi})-U_{ws,n}(\theta_{0},\pi)\right]$, we have

$$\Sigma=-\mathbb{E}\left[\frac{\Delta_{i}[\Delta_{i}-\pi(Y_{i},\mathbf{S}^{\prime}_{i})]}{\pi^{2}(Y_{i},\mathbf{S}^{\prime}_{i})}\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]+o^{*}(1),$$

$${}=-\mathbb{E}\left[\frac{1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]+o^{*}(1),$$

where the notation $o^{*}(a_{n})$ denotes a matrix whose components are uniformly $o^{*}(a_{n})$. Finally,

$$\textrm{Var}\left[U_{ws,n}(\theta_{0},\hat{\pi})\right]=\mathbb{E}\left[\frac{\dot{l}_{i}(\theta_{0},\pi)\dot{l}_{i}(\theta_{0},\pi)^{T}}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right]+\mathbb{E}\left[\frac{[1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})]\left[\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\right]$$

$${}-2\mathbb{E}\left[\frac{1-\pi(Y_{i},\mathbf{S}^{\prime}_{i})}{\pi(Y_{i},\mathbf{S}^{\prime}_{i})}\dot{l}_{i}^{*}(\theta_{0})\dot{l}_{i}^{*}(\theta_{0})^{T}\right]+o^{*}(1),$$

$${}=\Omega_{3}(\theta_{0},\pi)-\left[\Omega_{4}(\theta_{0},\pi)-\Omega_{5}(\theta_{0},\pi)\right]+o^{*}(1).$$

Thus, by the central limit theorem, we have $U_{ws,n}(\theta_{0},\hat{\pi})$ converges in distribution to the Gaussian vector of mean zero and variance $\Omega_{3}(\theta_{0},\pi)-\left[\Omega_{4}(\theta_{0},\pi)-\Omega_{5}(\theta_{0},\pi)\right]$. Because $\left[-\bar{U}_{ws,n}^{-1}(\theta_{0},\hat{\pi})-\Sigma^{-1}(\theta_{0})\right]$ converges in probability to $0$, by Slutsky’s theorem $\left[-\bar{U}_{ws,n}^{-1}(\theta_{0},\hat{\pi})-\Sigma^{-1}(\theta_{0})\right]U_{ws,n}(\theta_{0},\hat{\pi})$ converges in distribution to $0$.

Finally, by Lemma 4 and Slutsky’s theorem, $\sqrt{n}(\hat{\theta}^{ws}_{n}-\theta_{0})$ converges in distribution to the Gaussian vector of mean zero and variance

$$\Delta_{ws}:=\Sigma(\theta_{0})^{-1}\{\Omega_{3}(\theta_{0},\pi)-\left[\Omega_{4}(\theta_{0},\pi)-\Omega_{5}(\theta_{0},\pi)\right]\}[\Sigma(\theta_{0})^{-1}]^{T}.$$