A propensity score adjustment method for longitudinal time series models under nonignorable nonresponse

Abstract

Analysis of data with nonignorable nonresponse is an important and challenging task. Although some methods have been developed for inference under nonignorable nonresponse, they are only available for independent data. In this paper, we develop a two-stage propensity score adjustment method to estimate longitudinal time series models with nonignorable missingness. In particular, the response probability or propensity score is first estimated via solving the mean score equation based on the observed sample. Then, the inverse propensity scores are employed to conduct weighting adjustment for a composite likelihood based estimation. The propensity scores weighted estimation equations are shown to yield consistent and asymptotic normal estimators. Simulation studies and application to AIDS Clinical Trial data are presented to evaluate the performance of the proposed method.

Appendix

Proof of the response model identifiability. For the response model identifiability, the mean score function $\bar{S}(\eta )$ is not identifiable if there exist different $\eta$ and $\eta '$ such that $\bar{S}(\eta )=\bar{S}(\eta ')$ for all $x_{i}$, $\delta _{i,t}$ and $y_{i,t}$. To be specific, the mean score function $\bar{S}(\eta )$ in (3.7) consists of two parts, $\sum \nolimits _{i=1}^{n}\sum \nolimits _{t=2}^{T}\delta _{i,t}s_{y_{i,t}}(\eta )$ and

$$\begin{aligned} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}(1-\delta _{i,t})\frac{\int \pi ^{-1}_{y}(\eta )\hat{f}(y|x_{i},\delta _{i,t}=1)\partial \pi _{y}(\eta )/\partial \eta dy}{\int (\pi ^{-1}_{y}(\eta )-1)\hat{f}(y|x_{i},\delta _{i,t}=1)dy} \,. \end{aligned}$$

(A1)

The first part is not identifiable if and only if two different $\eta$ and $\eta '$ give the same $s_{y_{i,t}}(\eta )=\{\delta _{i,t}-\pi _{i,t}(\eta )\}\pi ^{-1}_{i,t}(\eta )\{1-\pi _{i,t}(\eta )\}^{-1}\partial \pi _{i,t} (\eta )/\partial \eta$ for all possible values of $x_{i}$, $\delta _{i,t}$ and $y_{i,t}$, where $\pi _{i,t} (\eta )$ is defined by (2.2). Here, we take the derivative of $\pi _{i,t}(\eta )$ with respect to $\eta _{1}$ for example, then we have

$$\begin{aligned} s_{y_{i,t}}(\eta )&=\{\delta _{i,t}-\pi _{i,t}(\eta )\}\pi ^{-1}_{i,t}(\eta )\{1-\pi _{i,t}(\eta )\}^{-1}\partial \pi _{i,t} (\eta )/\partial \eta _{1} \nonumber \\&= \frac{x_{i}[\delta _{i,t}\exp (-\eta _{1}x_{i}-\eta _{2}y_{i,t})+\delta _{i,t}-1]}{1+\exp (-\eta _{1}x_{i}-\eta _{2}y_{i,t})} \,. \end{aligned}$$

(A2)

Suppose that there exist that $\eta =(\eta _{1}, \eta _{2})$ and $\eta '=(\eta '_{1}, \eta '_{2})$ such that

$$\begin{aligned} \frac{x_{i}[\delta _{i,t}\exp (-\eta _{1}x_{i}-\eta _{2}y_{i,t})+\delta _{i,t}-1]}{1+\exp (-\eta _{1}x_{i}-\eta _{2}y_{i,t})}&=\frac{x_{i}[\delta _{i,t}\exp (-\eta '_{1}x_{i}-\eta '_{2}y_{i,t})+\delta _{i,t}-1]}{1+\exp (-\eta '_{1}x_{i}-\eta '_{2}y_{i,t})} \end{aligned}$$

(A3)

for all values of $x_{i}$, $\delta _{i,t}$ and $y_{i,t}$. It is easy to see that $\eta '_{1}=\eta _{1}$ must hold. Also, if $\eta '_{1}=\eta _{1} \ne 0$, then by taking $x_{i}$ sufficiently large or negative, we have $\delta _{i,t}\exp (-\eta _{1}x_{i}-\eta _{2}y_{i,t})+\delta _{i,t}-1=\delta _{i,t}\exp (-\eta '_{1}x_{i}-\eta '_{2}y_{i,t})+\delta _{i,t}-1$. It follows that $\eta '_{2}=\eta _{2}$. That is, the first part is identifiable.

The second part in (A1) is not identifiable if and only if two different $\eta$ and $\eta '$ give the same $\pi ^{-1}_{y}(\eta )\partial \pi _{y}(\eta )/\partial \eta = \pi ^{-1}_{y}(\eta ')\partial \pi _{y}(\eta ')/\partial \eta$ and $\pi ^{-1}_{y}(\eta )-1=\pi ^{-1}_{y}(\eta ')-1$ for all possible values of $x_{i}$, $\delta _{i,t}$ and $y_{i,t}$, because $\hat{f}(y|x_{i},\delta _{i,t}=1)$ in (A1) is obtained by (3.6) and not related to the parameter $\eta$. Similar to the discussion in the first part, we have that two different $\eta$ and $\eta '$ cannot produce the equal second part. That is, the second part is also identifiable. Thus, the response model parameter $\eta$ estimated by the mean score equation in (3.7) is identifiable.

Proof of Lemma 1

Note that

$$\begin{aligned} \mathcal {I}_{21}(\theta , \eta )=&-E\bigg \{ \frac{\partial u(\theta ; \eta )}{\partial \eta }\bigg \}\nonumber \\ =&-E\bigg \{ \sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\frac{\partial \delta _{t-k}\delta _{t}\pi ^{-1}_{t-k}(\eta )\pi ^{-1}_{t}(\eta )U(\theta ;y_{t-k},y_{t})}{\partial \eta }\bigg \}\nonumber \\ =&E\bigg \{\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T} \frac{\delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t})\partial \pi _{t-k}(\eta )/ \partial \eta }{\pi ^{2}_{t-k}(\eta )\pi _{t}(\eta )} \nonumber \\&+ \sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\frac{ \delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t}) \partial \pi _{t}(\eta )/ \partial \eta }{\pi _{t-k}(\eta )\pi ^{2}_{t}(\eta )}\bigg \} \nonumber \\ =&E\bigg \{ \sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\frac{\delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t})(1-\pi _{t-k}(\eta ))z(x,y_{t-k};\eta )}{\pi _{t-k}(\eta )\pi _{t}(\eta )}\nonumber \\&+\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T} \frac{\delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t})(1-\pi _{t}(\eta ))z(x,y_{t};\eta )}{\pi _{t-k}(\eta )\pi _{t}(\eta )}\bigg \} \nonumber \\ =&E\bigg \{ \sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\frac{ \delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t})(\delta _{t-k}-\pi _{t-k}(\eta ))z(x,y_{t-k};\eta )}{\pi _{t-k}(\eta )\pi _{t}(\eta )} \nonumber \\&+ \sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\frac{ \delta _{t-k}\delta _{t}U(\theta ;y_{t-k},y_{t})(\delta _{t}-\pi _{t}(\eta ))z(x,y_{t};\eta )}{\pi _{t-k}(\eta )\pi _{t}(\eta )}\bigg \} \nonumber \\ =&E\Big \{\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\delta _{t-k}\delta _{t}\pi ^{-1}_{t-k}(\eta )\pi ^{-1}_{t}(\eta )U(\theta ;y_{t-k},y_{t})\nonumber \\&\cdot \big [ s^{T}(\eta ;\delta _{t-k},x,y_{t-k})+ s^{T}(\eta ;\delta _{t},x,y_{t}) \big ]\Big \}\,, \end{aligned}$$

(A4)

where $z(x,y_{t};\eta )=\pi ^{-1}_{t}(\eta )\{1-\pi _{t}(\eta )\}^{-1}\partial \pi _{t}(\eta )/\partial \eta$, yielding (4.1). $\square$

Since $u(\theta ; \eta )= \sum \nolimits _{k=1}^{K}\sum \nolimits _{t=k+1}^{T}\delta _{t-k}\delta _{t}\pi ^{-1}_{t-k}(\eta )\pi ^{-1}_{t}(\eta )U(\theta ;y_{t-k},y_{t})$, and the partial derivatives $\partial U(\theta ;y_{t-k},y_{t})/ \partial \theta ^{T}$ of $U(\theta ; y_{t-k},y_{t})$ with respect to $\theta$ exist for $k=1,\ldots , K$, the equation (4.2) obviously holds.

Proof of Theorem 1

Taking expectation on $\bar{S}(\eta )$, we have

$$\begin{aligned} E(\bar{S}(\eta ))= & {} E\Big \{\sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\big \{\delta _{i,t}s(\eta ;\delta _{i,t},x_{i},y_{i,t}) \nonumber \\&+(1-\delta _{i,t})E[s(\eta ;\delta _{i,t},x_{i},Y_{t})|x_{i},\delta _{i,t}=0]\big \}\Big \} \nonumber \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\Big \{ E[\delta _{i,t}s(\eta ;\delta _{i,t},x_{i},y_{i,t})] \nonumber \\&+ E\{(1-\delta _{i,t})E[s(\eta ;\delta _{i,t},x_{i},Y_{t})|x_{i},\delta _{i,t}=0]\} \Big \}\nonumber \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\Big \{ E[\delta _{i,t}\{\delta _{i,t}-\pi (x_{i},y_{i,t};\eta )\}z(x_{i},y_{i,t};\eta )]\nonumber \\&+ E\{(1-\delta _{i,t})E[\{\delta _{i,t}-\pi (x_{i},Y_{t};\eta )\}z(x_{i},Y_{t};\eta )|x_{i},\delta _{i,t}=0]\} \Big \}\nonumber \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\Big \{ E[\pi (x_{i},y_{i,t};\eta )(1-\pi (x_{i},y_{i,t};\eta ))z(x_{i},y_{i,t};\eta )]\nonumber \\&- E\{(1-\delta _{i,t})E[(\pi (x_{i},Y_{t};\eta ))z(x_{i},Y_{t};\eta )|x_{i}]\} \Big \}\nonumber \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\Big \{ E[\partial \pi (x_{i},y_{i,t};\eta )/\partial \eta ]\nonumber \\&- E\{E[(1-\pi (x_{i},Y_{t};\eta ))\pi (x_{i},Y_{t};\eta )z(x_{i},Y_{t};\eta )|x_{i}]\} \Big \} \nonumber \\= & {} \sum \limits _{i=1}^{n}\sum \limits _{t=2}^{T}\Big \{E[\partial \pi (x_{i},y_{i,t};\eta )/\partial \eta ]- E[\partial \pi (x_{i},Y_{t};\eta )/\partial \eta ] \Big \}=0\,. \end{aligned}$$

(A5)

That is, $\bar{S}(\eta )$ is unbiased. It follows from standard asymptotic theory that $\widehat{\eta }$ is a consistent estimator of $\eta _{0}$. $\square$

In fact, $U_{PS}(\theta ; \widehat{\eta })$ is also unbiased because

$$\begin{aligned} E\Big \{U_{PS}(\theta ; \widehat{\eta })\Big \} =&E\Big \{\sum \limits _{i=1}^{n}\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\delta _{i,t-k}\delta _{i,t}\pi ^{-1}_{i,t-k}(\widehat{\eta })\pi ^{-1}_{i,t}(\widehat{\eta })U(\theta ;y_{i,t-k},y_{i,t}) \Big \}\nonumber \\ =&\, E\Big \{\sum \limits _{i=1}^{n}\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}E\big [\delta _{i,t-k}\delta _{i,t}\pi ^{-1}_{i,t-k}(\widehat{\eta })\pi ^{-1}_{i,t}(\widehat{\eta })\nonumber \\&\quad U(\theta ;y_{i,t-k},y_{i,t})| x_{i}, y_{i,t-k},y_{i,t}\big ] \Big \}\nonumber \\ =&\, E\Big \{\sum \limits _{i=1}^{n}\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}\pi ^{-1}_{i,t-k}(\eta _{0})\pi ^{-1}_{i,t}(\eta _{0})U(\theta ;y_{i,t-k},y_{i,t})\nonumber \\&\quad E\big [\delta _{i,t-k}\delta _{i,t}| x_{i}, y_{i,t-k},y_{i,t}\big ] \Big \}\nonumber \\ =&\, E\Big \{\sum \limits _{i=1}^{n}\sum \limits _{k=1}^{K}\sum \limits _{t=k+1}^{T}U(\theta ;y_{i,t-k},y_{i,t}) \Big \}\nonumber \\ =&\, E\Big \{\sum \limits _{k=1}^{K}U(\theta ;y_{t-k},y_{t}) \Big \}=0\,. \end{aligned}$$

(A6)

Therefore, $\widehat{\theta }_{PS}$ is a consistent estimator of $\theta _{0}$.

Proof of Theorem 2

Let $s(\eta )=\sum \limits _{t=2}^{T}\big \{\delta _{t}s(\eta ;\delta _{t},x,y_{t})+(1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\big \}$ and

$$\begin{aligned}&\dot{s}(\eta ;\delta _{t},x,y_{t}) = \partial s(\eta ;\delta _{t},x,y_{t})/ \partial \eta ^{T} \nonumber \\&\quad = (\delta _{t}-\pi (x,y_{t};\eta ))\frac{ \partial z(x,y_{t};\eta )}{\partial \eta ^{T}}- z(x,y_{t};\eta )\frac{ \partial \pi (x,y_{t};\eta )}{\partial \eta ^{T}}\,. \end{aligned}$$

(A7)

We have

$$\begin{aligned} \frac{\partial s(\eta )}{\partial \eta ^{T}} = \sum \limits _{t=2}^{T}\bigg \{\delta _{t} \dot{s}(\eta ;\delta _{t},x,y_{t}) + (1-\delta _{t}) \frac{\partial E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]}{\partial \eta ^{T}}\bigg \}\,. \end{aligned}$$

(A8)

Since

$$\begin{aligned} E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]=\frac{\int s(\eta ;\delta _{t},x,y)f(y|x,\delta _{t}=1)O(x,y;\eta )dy}{\int O(x,y;\eta )f(y|x,\delta _{t}=1)dy}\,, \end{aligned}$$

(A9)

the derivative $\partial E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]/ \partial \eta ^{T}$ in the Eq. (A8) can be written as

$$\begin{aligned} \frac{\partial E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]}{\partial \eta ^{T}}=&\frac{\int \dot{s}(\eta ;\delta _{t},x,y)\hat{f}(y|x,\delta _{t}=1)O(x,y;\eta )dy}{\int O(x,y;\eta )\hat{f}(y|x,\delta _{t}=1)dy}\nonumber \\&+ \frac{\int s(\eta ;\delta _{t},x,y)\hat{f}(y|x,\delta _{t}=1) \partial O(x,y;\eta )/ \partial \eta ^{T} dy}{\int O(x,y;\eta )\hat{f}(y|x,\delta _{t}=1)dy}\nonumber \\&- \frac{\int s(\eta ;\delta _{t},x,y)\hat{f}(y|x,\delta _{t}=1)O(x,y;\eta )dy}{\int O(x,y;\eta )\hat{f}(y|x,\delta _{t}=1)dy} \nonumber \\&\cdot \frac{\int \hat{f}(y|x,\delta _{t}=1)\partial O(x,y;\eta )/ \partial \eta ^{T} dy}{\int O(x,y;\eta )\hat{f}(y|x,\delta _{t}=1)dy}\,. \end{aligned}$$

(A10)

By $\partial O(x,y_{t};\eta )/ \partial \eta ^{T} = - (\partial \pi (x,y_{t};\eta )/ \partial \eta ^{T}) / \pi ^{2}(x,y_{t};\eta ) = - O(x,y_{t};\eta )z(x,y_{t};\eta )$, we obtain

$$\begin{aligned}&\frac{\partial E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]}{\partial \eta ^{T}} = E[\dot{s}(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0] \nonumber \\&\quad - E[s(\eta ;\delta _{t},x,Y_{t})z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0]\nonumber \\&\quad + E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\cdot E[z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \,. \end{aligned}$$

(A11)

Combining equations (A8) and (A11), we have

$$\begin{aligned} E\bigg (\frac{\partial s(\eta )}{\partial \eta ^{T}} \bigg )=&E \bigg \{\sum \limits _{t=2}^{T}\Big \{ \delta _{t} \dot{s}(\eta ;\delta _{t},x,y_{t}) + (1-\delta _{t})E[\dot{s}(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\nonumber \\&- (1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \nonumber \\&+ (1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\cdot E[z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \Big \}\bigg \}\nonumber \\ =&\, E \Big \{ \sum \limits _{t=2}^{T}\big \{\delta _{t} \dot{s}(\eta ;\delta _{t},x,y_{t}) + (1-\delta _{t})E[\dot{s}(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\big \}\Big \}\nonumber \\&- E \Big \{\sum \limits _{t=2}^{T}(1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \Big \}\nonumber \\&+ E \Big \{\sum \limits _{t=2}^{T}(1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\cdot E[z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \Big \} \,. \end{aligned}$$

(A12)

Since

$$\begin{aligned}&E \Big \{ \sum \limits _{t=2}^{T}\big \{\delta _{t} \dot{s}(\eta ;\delta _{t},x,y_{t}) + (1-\delta _{t})E[\dot{s}(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\big \}\Big \} \nonumber \\&\quad = E \Big \{ \sum \limits _{t=2}^{T}\big \{ \pi (x,y_{t};\eta ) \dot{s}(\eta ;\delta _{t},x,y_{t}) + (1-\pi (x,y_{t};\eta ))\dot{s}(\eta ;\delta _{t},x,y_{t})\big \}\Big \}\nonumber \\&\quad = E \Big [\sum \limits _{t=2}^{T}\dot{s}(\eta ;\delta _{t},x,y_{t})\Big ] \end{aligned}$$

(A13)

and

$$\begin{aligned}&E \Big \{\sum \limits _{t=2}^{T}(1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0]\Big \}\nonumber \\&\quad = - E \Big \{\sum \limits _{t=2}^{T}(1-\delta _{t})E[\pi (x,Y_{t};\eta )z(x,Y_{t};\eta ) z^{T}(x,Y_{t};\eta )|x]\Big \}\nonumber \\&\quad = - E \Big \{E\big [\sum \limits _{t=2}^{T}(1-\pi (x,Y_{t};\eta ))\pi (x,Y_{t};\eta )z(x,Y_{t};\eta ) z^{T}(x,Y_{t};\eta )|x\big ]\Big \}\nonumber \\&\quad = - E\Big [\sum \limits _{t=2}^{T}z(x,Y_{t};\eta ) \partial \pi (x,y_{t};\eta ))/ \partial \eta ^{T}\Big ]\nonumber \\&\quad = E \Big [\sum \limits _{t=2}^{T}\dot{s}(\eta ;\delta _{t},x,y_{t})\Big ] \,, \end{aligned}$$

(A14)

we then have

$$\begin{aligned} E\bigg (\frac{\partial s(\eta )}{\partial \eta ^{T}} \bigg ) =&E \Big \{\sum \limits _{t=2}^{T}(1-\delta _{t})E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]\cdot E[z^{T}(x,Y_{t};\eta ) |x,\delta _{t}=0] \Big \} \,. \end{aligned}$$

(A15)

Let $\mathcal {I}_{11}(\eta )= - E\big (\partial s(\eta )/ \partial \eta ^{T} \big )$. Since $\widehat{\eta }$ is the solution to the mean score equation $\bar{S}(\eta )=0$ in (3.3), we have

$$\begin{aligned} \sqrt{n}\left( \widehat{\eta }-\eta _{0} \right) \xrightarrow {\ d\ }N\left( 0, \mathcal {I}^{-1}_{11}(\eta _{0}) \right) \,, \end{aligned}$$

(A16)

where

$$\begin{aligned} \mathcal {I}^{-1}_{11}(\eta _{0}) =&-\bigg \{E\Big [\sum \limits _{t=2}^{T}(1-\delta _{t})E[s(\eta _{0};\delta _{t},x,Y_{t})|x,\delta _{t}=0]E[z^{T}(x,Y_{t};\eta _{0})|x,\delta _{t}=0]\Big ]\bigg \}^{-1} \nonumber \\ =&-\bigg \{E\Big [\sum \limits _{t=2}^{T}(1-\delta _{t})\bar{s}_{0}(\eta _0;x)\bar{z}_{0}(\eta _0;x)\Big ]\bigg \}^{-1} \,, \end{aligned}$$

(A17)

$\bar{s}_{0}(\eta ;x)=E[s(\eta ;\delta _{t},x,Y_{t})|x,\delta _{t}=0]$ and $\bar{z}_{0}(\eta ;x)=E[z^{T}(x,Y_{t};\eta )|x,\delta _{t}=0]$, completing the proof of Theorem 2. $\square$

Proof of Theorem 3

Since $\widehat{\psi } = (\widehat{\eta }, \widehat{\theta }_{PS})$ is the solution to

$$\begin{aligned} \left( \begin{array}{c} \bar{S}(\eta )=0\\ U_{PS}(\theta ; \eta )=0 \end{array}\right) =\left( \begin{array}{c} 0\\ 0 \end{array} \right) \,, \end{aligned}$$

(A18)

the variance of $\widehat{\psi } = (\widehat{\eta }, \widehat{\theta }_{PS})$ can be obtained by

$$\begin{aligned} \mathrm{var}\left\{ \sqrt{n}\left( \begin{array}{c} \widehat{\eta }-\eta _{0}\\ \widehat{\theta }_{PS}-\theta _{0} \end{array}\right) \right\} = \mathcal {I}^{-1} \mathrm{var}\left\{ \begin{array}{c} s(\eta _{0})\\ u(\theta _{0}; \eta _{0}) \end{array}\right\} \mathcal {I}^{-T} \,, \end{aligned}$$

(A19)

where $\mathcal {I}=\mathcal {I}(\theta _{0}, \eta _{0})$ and

$$\begin{aligned} \mathcal {I}(\theta , \eta ) = \left( \begin{array}{cc} \mathcal {I}_{11}(\eta )&{} 0\\ \mathcal {I}_{21}(\theta ,\eta )&{} \mathcal {I}_{22}(\theta ,\eta ) \end{array}\right) = -\left\{ \begin{array}{cc} E\big (\frac{\partial s(\eta )}{\partial \eta ^{T}}\big )&{} E\big (\frac{\partial s(\eta )}{\partial \theta ^{T}}\big )\\ E\big (\frac{\partial u(\theta ; \eta )}{\partial \eta ^{T}}\big ) &{} E\big (\frac{\partial u(\theta ; \eta )}{\partial \theta ^{T}}\big ) \end{array}\right\} \,. \end{aligned}$$

(A20)

Further, we have

$$\begin{aligned} \mathcal {I}^{-1}= \left( \begin{array}{cc} \mathcal {I}^{-1}_{11}&{} 0\\ -\mathcal {I}^{-1}_{22}\mathcal {I}_{21}\mathcal {I}^{-1}_{11}&{} \mathcal {I}^{-1}_{22} \end{array}\right) \,, \end{aligned}$$

(A21)

where $\mathcal {I}_{11} = \mathcal {I}_{11}(\eta _{0})$, $\mathcal {I}_{22} = \mathcal {I}_{22}(\theta _{0}, \eta _{0})$, $\mathcal {I}_{21} = \mathcal {I}_{21}(\theta _{0}, \eta _{0})$. Combining equations (A19) and (A21), we have

$$\begin{aligned} \Sigma _{\theta } =&\mathrm{var}(\sqrt{n}(\widehat{\theta }_{PS}-\theta _{0}))= \mathrm{var}\big \{\mathcal {I}^{-1}_{22}[u(\theta _{0}; \eta _{0})-\mathcal {I}_{21}\mathcal {I}^{-1}_{11}s(\eta _{0})]\big \}\nonumber \\ =&\mathcal {I}^{-1}_{22} \mathrm{var}\big [u(\theta _{0}; \eta _{0})-\mathcal {I}_{21}\mathcal {I}^{-1}_{11}s(\eta _{0})\big ] \mathcal {I}^{-T}_{22} \,. \end{aligned}$$

(A22)

It follows that

$$\begin{aligned} \sqrt{n}\left( \widehat{\theta }_{PS}-\theta _{0} \right) \xrightarrow {\ d\ }N\left( 0, \Sigma _{\theta } \right) \,, \end{aligned}$$

(A23)

as $n \rightarrow \infty$, where

$$\begin{aligned} \Sigma _{\theta }= \mathcal {I}^{-1}_{22} \mathrm{var}[u(\theta _{0}; \eta _{0})- \mathcal {I}_{21}\mathcal {I}^{-1}_{11}s(\eta _{0})]\mathcal {I}^{-T}_{22}\,. \end{aligned}$$

That is, Theorem 3 holds. $\square$