Skip to main content
Log in

Nonresponse adjusted estimation based on a composite weighting method in a panel survey

  • Research Article
  • Published:
Journal of the Korean Statistical Society Aims and scope Submit manuscript

Abstract

Respondents to panel surveys are commonly divided into continuous and noncontinuous groups based on their response patterns. In this study, we propose an estimator based on composite weights as an effective nonresponse adjustment method to reduce the bias of noncontinuous response groups. We derive the properties of the proposed estimator, such as its bias and mean squared error, and then compare its efficiency with that of alternative estimators. We present the results of simulations demonstrating that the proposed estimator exhibits less variance than the conventional method of directly using the response rate of a noncontinuous response group. It also exhibited a lower bias than that obtained using the response rate of the continuous response group. The composite weighting method used in the proposed estimator showed stable results in terms of minimizing extreme weights, indicating that it may be considered highly effective for noncontinuous response groups in panel surveys.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

Data availability

1. HRS website https://hrsdata.isr.umich.edu.

2. See the following for SIPP information- response patterns : Kalton G., Lepkowski, J., Heeringa, S., Lin, T. K. and Miller, M. E. (1987), The treatment of person-wave nonresponse in longitudinal surveys, Ch1. table1, Ch3. table1, Working Paper No. SEHSD-WP1987-04 OR SIPP-WP-26, Census Beureau of U.S.

3. KLoSA website: https://survey.keis.or.kr/klosa/klosa01.jsp

References

  • Choi, H. A. (2021). Nonresponse adjusted estimation based on composite weighting method in panel survey. A Doctoral Dissertation, Sookmyung Women’s University.

  • Choi, H. A., & Kim, Y. W. (2019). Composite estimation type weighting adjustment for bias reduction of non-continuous response group in panel survey. The Korean Journal of Applied Statistics, 32(3), 375–389.

    Google Scholar 

  • James, B. G., David, B., James, N., Anni, O. & Andrew, S. (2018). The dynamics of aging: evidence from the English longitudinal study of aging 2002-16(Wave8) Ch5. National Center for Social Research.

  • Kim, Y. W., Lee, K. J., & Park, I. H. (2015). Report of recreating weights for KLoSA wave 1-4 and weights for Wave 5. Korea Employment Information Service.

  • Lee, J. R., Choi, E. Y., Do, N. H., Song, S. Y., Wang, Y. H., & Jung, Y. H. (2011). Panel study on Korean Children 2011 Report. Korea Institute of Child Care and Education

  • Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data. Hoboken: Wiley-Interscience.

    Book  MATH  Google Scholar 

  • Little, R. J. A. (1995). Modelling the drop-out mechanism in repeated-measures studies. Journal of the American Statistical Association, 90(431), 1112–1121.

    Article  MathSciNet  MATH  Google Scholar 

  • Park, M. G., & Kim, Y. W. (2013). Research report of YP wave 6 and new response model development. Korea Employment Information Service.

  • Park, M. G., Kim, Y. W., & Byun, J. S. (2013). Weights research of Korean labor & income panel survey. Korea Labor Institute

  • Park, M. G., Lee, K. S., Park, H. S., & Kang, H. C. (2011). A study on the construction of weights for KYPS. The Korean Association for Survey Research, 12(3), 173–186.

    Google Scholar 

  • Qian, Y., & Xie, H. (2009). Measuring the impact of nonignorable non-response in panel data with non-monotone missingness. Journal of Applied Econometrics, Forthcoming.

  • Rao, J. N. K. (2003). Small area estimation, CH2-Ch4. John Wiley & Sons, Ltd.

    Book  Google Scholar 

  • Sadi, H. (2014). Weighting for non-monotonic response pattern in longitudinal surveys. Institute for Social and Economic Research No.2014-34, University of Essex.

  • Sarndal, C. E., Swensson, B., & Wretman, J. (1992). Model assisted survey sampling. Springer Verlag New York, Inc.

    Book  MATH  Google Scholar 

  • Seaman, S. R., White, I. R., Copas, A. J., & Li, L. (2011). Combining multiple imputation and inverse-probability weighting. Biometrics, 68, 129–137.

    Article  MathSciNet  MATH  Google Scholar 

  • Watson, N., & Wooden, M. (2014). Re-engaging with survey non-respondents: Evidence from three house hold panels. Journal of the Royal Statistical Society Series A (Statistics in Society), 77(2), 499–522.

    Article  Google Scholar 

  • Zao, Y. (2020). Statistical inference for missing data mechanisms. Statistics in Medicine, 39(28), 4325–4333.

    Article  MathSciNet  Google Scholar 

  • Zhao, P., Wang, L., & Shao, J. (2017). Analysis of longitudinal data under nonignorable nonmonotone nonresponse. Statistics and its Interface, 11(2).

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Young-Won Kim.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

(1) Proof of Theorem 1

According to the theory of two-phase sampling, the bias of the proposed estimator \({\hat{t}}_{y_{t}^{*}}\) is derived as follows. The estimator in the C group is an unbiased estimator under Assumptions A.2 and A.5, where

$$\begin{aligned}& E_{r}\left({\hat{N}}_{t}^{(C)} | s_{t}^{(C)}\right)=\sum ^{}_{s_{t}^{(C)}}\left(\frac{E_{r}(R_{i,t} | s_{t}^{(C)})}{\pi _{i}\phi _{i,t}^{(C)}}\right) =\sum ^{}_{s_{t}^{(C)}}\frac{1}{\pi _{i}}={\hat{N}}_{t}^{(C,1)},\\& E_{r}\left(\frac{1}{{\hat{N}}_{t}^{(C)}}|s_{t}^{(C)}\right) \approx \frac{1}{E_{r}\left({\hat{N}}_{t}^{(C)}|s_{t}^{(C)}\right)}=\frac{1}{{\hat{N}}_{t}^{(C,1)}},\\ \end{aligned}$$
$$\begin{aligned}&E_{D}E_{r}\left({\hat{t}}_{y_{t}^{(C)}}-t_{y_{t}^{(C)}}|s_{t}^{(C)}\right)\\ &=E_{D}\left[{\hat{N}}_{t}^{(C,1)}E_{r}\left(\frac{1}{{\hat{N}}_{t}^{(C)}}|s_{t}^{(C)}\right)\sum ^{}_{s_{t}^{(C)}} \frac{E_{r}(R_{i,t}|s_{t}^{(C)})}{\pi _{i}\phi _{i,t}^{(C)}}y_{i,t}\right]-t_{y_{t}^{(C)}}\\&\approx E_{D}\left(\sum ^{}_{s_{t}^{(C)}}\frac{y_{i,t}}{\pi _{i}}\right)-t_{y_{t}^{(C)}} =0. \end{aligned}$$

Thus, the bias of the proposed estimator \({\hat{t}}_{y_{t}^{*}}\) is that of the estimator \({\hat{t}}_{y_{t}^{(NC)^{*}}}\) of the NC group,

$$\begin{aligned}E\left({\hat{t}}_{y_{t}^{*}}-t_{y_{t}}\right)&=E_{D}E_{r}\left({\hat{t}}_{y_{t}^{*}}-t_{y_{t}}|s_{t}^{(NC)}\right) \\&=E_{D}E_{r}\left({\hat{t}}_{y_{t}^{(NC)^{*}}}-t_{y_{t}^{(NC)}}|s_{t}^{(NC)}\right)\\&=E_{D}\left[\sum ^{}_{s_{t}^{(NC)}}E_{r}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}}|s_{t}^{(NC)}\right) \frac{E_{r}(R_{i,t}|s_{t}^{(NC)})y_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}-\sum ^{}_{s_{t}^{(NC)}}\frac{y_{i,t}}{\pi _{i}}\right]\\&\approx E_{D}\left[\sum ^{}_{s_{t}^{(NC)}}\frac{y_{i,t}}{\pi _{i}}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{E_{r}\left({\hat{N}}_{t}^{(NC)^{*}}|s_{t}^{(NC)}\right)} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1\right)\right]\\&\approx E_{D}\left[\sum ^{}_{s_{t}^{(NC)}}\frac{y_{i,t}}{\pi _{i}}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1\right)\right], \end{aligned}$$

where

$$\begin{aligned} E_{r}\left(\frac{1}{{\hat{N}}_{t}^{(NC)^{*}}}|s_{t}^{(NC)}\right)&\approx \frac{1}{E_{r}({\hat{N}}_{t}^{(NC)^{*}}|s_{t}^{(NC)})}\\&=1/\sum ^{}_{s_{t}^{(NC)}}\frac{\phi _{i,t}^{(NC)}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}\\&=1/N_{t}^{(NC)^{*}}. \end{aligned}$$

Here, \(E_{D}\) is a first-phase sample expectation based on the sampling design. Given the first-phase sample, the expression in \(E_{D}\) is the bias of the estimator \({\hat{t}}_{y_{t}^{NC^{*}}}\) based on whether a response at wave t exists in the NC group. \(\square\)

(2) Proof of Theorem 2

$$\begin{aligned} MSE({\hat{t}}_{y_{t}^{*}})&=E({\hat{t}}_{y_{t}^{*}}-E({\hat{t}}_{y_{t}^{*}})+E({\hat{t}}_{y_{t}^{*}})-t_{y_{t}})^{2} \\ {}&=E({\hat{t}}_{y_{t}^{*}}-E({\hat{t}}_{y_{t}^{*}}))^{2}+E(E({\hat{t}}_{y_{t}^{*}})-t_{y_{t}})^{2} \\ {}&=E({\hat{t}}_{y_{t}^{*}}-E({\hat{t}}_{y_{t}^{*}}))^{2}+(E({\hat{t}}_{y_{t}^{*}})-t_{y_{t}})^{2} \\ {}&=V({\hat{t}}_{y_{t}^{*}})+(E_{D}E_{r}({\hat{t}}_{y_{t}^{*}}|s)-t_{y_{t}})^{2} \\ {}&=V({\hat{t}}_{y_{t}^{*}})+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{*}}|s)\}^{2} \\ {}&=V({\hat{t}}_{y_{t}^{(C)}}+{\hat{t}}_{y_{t}^{(NC)^{*}}})+2Cov({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}}) \\ {}&+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{(C)}}+{\hat{t}}_{y_{t}^{(NC)^{*}}}|s)\}^{2} \\ {}&=V({\hat{t}}_{y_{t}^{(C)}})+V({\hat{t}}_{y_{t}^{(NC)^{*}}})+2Cov({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}}) \\ {}&+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})\}^{2}+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)})\}^{2}. \end{aligned}$$

The variance and covariance based on the response group are given as follows when we regard the respondent sample at wave t as a second-phase sampling.

$$\begin{aligned}&V({\hat{t}}_{y_{t}^{(C)}})=V_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})+E_{D}V_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})\\&V({\hat{t}}_{y_{t}^{(NC)^{*}}})=V_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)})+E_{D}V_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)})\\&2Cov({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}})=2\{Cov_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}}) +E_{D}Cov_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}})\}. \end{aligned}$$

If we substitute the variance and covariance expression into the above, the following expression holds.

$$\begin{aligned}&MSE({\hat{t}}_{y_{t}^{*}})\\&=\{V_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}| i \in s_{t}^{(C)})+E_{D}V_{r}({\hat{t}}_{y_{t}^{(C)}} | i \in s_{t}^{(C)})+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})\}^{2}\} \\ {}&+\{V_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)})+E_{D}V_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)}) \\ {}&+\{E_{D}Bias_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)})\}^{2}\} \\ {}&+2\{Cov_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)*}})+E_{D}Cov_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)*}})\}. \end{aligned}$$

Given that \(\Delta _{ij|s_{t}^{(g)}}=V(I_{i}|s_{t}^{(C)},I_{j}|s_{t}^{(NC)})=\phi _{i,t}^{(C)}\phi _{j,t}^{(NC)}-\phi _{i,t}^{(C)}\phi _{j,t}^{(NC)}=0\), \(Cov_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}})=0.\)

$$\begin{aligned}&=\{V_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}| i \in s_{t}^{(C)})+E_{D}mse_{r}({\hat{t}}_{y_{t}^{(C)}}| i \in s_{t}^{(C)})\} \\&+\{V_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}} |i \in s_{t}^{(NC)})+E_{D}mse_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}} |i \in s_{t}^{(NC)})\} \\&+2Cov_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}},{\hat{t}}_{y_{t}^{(NC)^{*}}}), \end{aligned}$$
$$\begin{aligned}&=V_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})+V_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}} |i \in s_{t}^{(NC)}) \\ {}&+2Cov_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}, {\hat{t}}_{y_{t}^{(NC)^{*}}}) \\ {}&+E_{D}mse_{r}({\hat{t}}_{y_{t}^{(C)}}|i \in s_{t}^{(C)})+E_{D}mse_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|i \in s_{t}^{(NC)}). \qquad \square \end{aligned}$$

\(\square\)

(3) Proof of Theorem 3

We assume the group response rate \(\phi _{i,t}^{(g)}\) to be known. Because the group mean \(\tilde{y}_{t}^{(g)}\) is a ratio estimator, according to a Taylor linearization, the first order is roughly as follows (Sarndal et al., 1992).

$$\begin{aligned} \tilde{y}_{t}^{(g)}=&\frac{1}{{\hat{N}}_{t}^{(g)}}\sum ^{}_{i \in s^{(g)}}\frac{R_{i,t}y_{i,t}}{\pi _{i}\phi _{i,t}^{(g)}}\\&= \frac{1}{N_{t}^{(g)}}\left({\hat{t}}_{y_{t}}^{(g)}-\frac{{\hat{N}}_{t}^{(g)}}{N_{t}^{(g)}}{t}_{y_{t}}^{(g)}\right) + O(\frac{1}{N_{t}^{(g)2}}),\\&\approx \frac{1}{N_{t}^{(g)}}\left({\hat{t}}_{y_{t}}^{(g)}-{\hat{N}}_{t}^{(g)}{\bar{y}}_{u_{t}}^{(g)}\right),\quad {\hat{t}}_{y_{t}}^{(g)}=\sum ^{}_{s^{(g)}}\frac{R_{i,t}y_{i,t}}{\pi _{i}\phi _{i,t}^{(g)}},\quad {\hat{N}}_{t}^{(g)}=\sum ^{}_{s^{(g)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(g)}}, \end{aligned}$$

where \({\bar{y}}_{u_{t}}^{(g)}\) is unknown. We can apply \(\tilde{y}_{t}^{(g,1)}\) rather than \({\bar{y}}_{u_{t}}^{(g)}\) according to Assumptions A.2 and A.5.

$$\begin{aligned} E_{D}E_{r}\left(\tilde{y}_{t}^{(g)}|s_{t}^{(g)}\right)&=E_{D}(\tilde{y}_{t}^{(g,1)})\\&=E_{D}\left(\frac{1}{{\hat{N}}_{t}^{(g,1)}}\sum ^{}_{s_{t}^{(g)}}\frac{y_{i}}{\pi _{i}}\right)\\&={\bar{y}}_{u_{t}}^{(g)}. \end{aligned}$$

The estimator \({\hat{N}}_{t}^{(g)}\) is approximately unbiased for \({N}_{t}^{(g)}\). Therefore,

$$\begin{aligned} \tilde{y}_{t}^{(g)}&\approx \frac{1}{{\hat{N}}_{t}^{(g)}}\sum ^{}_{i \in s^{(g)}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(g)}}, \quad u_{i,t}=y_{i,t}-\tilde{y}_{t}^{(g,1)}. \end{aligned}$$

The Taylor linearization of the proposed estimator in the NC group is given as follows.

$$\begin{aligned} {\hat{t}}_{y_{t}^{(NC)^{*}}}&={\hat{N}}_{t}^{(NC,1)}\tilde{y}_{t}^{(NC)^{*}}\\&= \frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)}}\left({\hat{t}}_{y_{t}}^{(NC)*}-\frac{{\hat{N}}_{t}^{(NC)*}}{N_{t}^{(NC)}}{t}_{y_{t}}^{(NC)}\right) + O\left(\frac{1}{N_{t}^{(NC)2}}\right),\\&\approx \frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)}}\left({{\hat{t}}}_{{y_t}^{(NC)*\ }}-\frac{{{\hat{N}}}_t^{(NC)*}}{N_t^{(NC)}}t_{{y_t}^{(NC)\ }}\right) \end{aligned}$$

The estimator \({\hat{N}}_{t}^{(NC,1)}\) is approximately unbiased for \({N}_{t}^{(NC)}\) based on A.5. In addition, we can use the unbiased estimator \(\tilde{y}_{t}^{(NC,1)}\) for \({\bar{y}}_{u_{t}}^{(NC)}\). We can therefore express this as follows using \(\frac{{{\hat{N}}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)*}}\).

$$\begin{aligned}&\approx \frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}({{{\hat{N}}}_{t}^{(NC,1)}}/{{\hat{N}}_{t}^{(NC)*}})}\sum ^{}_{s_{t}^{(NC)}}\frac{{{\hat{N}}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)*}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}\\&= \frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}}\sum ^{}_{s_{t}^{(NC)}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}, \quad u_{i,t}=y_{i,t}-\tilde{y}_{t}^{(NC,1)}. \end{aligned}$$

Using the Taylor linearization rather than the group mean, we obtain the following.

$$\begin{aligned} V_{D}E_{r}&\left({\hat{t}}_{y_{t}^{(C)}} | i \in s_{t}^{(C)},\ R_{i,t}=1\right) +V_{D}E_{r}\left({\hat{t}}_{y_{t}^{(NC)^{*}}} | i \in s_{t}^{(NC)},\ R_{i,t}=1\right)\\&+2Cov_{D}E_{r}({\hat{t}}_{y_{t}^{(C)}}, {\hat{t}}_{y_{t}^{(NC)^{*}}}|R_{i,t}=1,R_{j,t}=1)\\&\approx V_{D}E_{r}\left({\hat{N}}_{t}^{(C,1)}\frac{1}{{\hat{N}}_{t}^{(C)}}\sum ^{}_{s_{t}^{(C)}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}| i \in s_{t}^{(C)}\right)\\&\quad + V_{D}E_{r}\left({\hat{N}}_{t}^{(NC,1)}\frac{1}{{\hat{N}}_{t}^{(NC)^{*}}}\sum ^{}_{s_{t}^{(NC)}} \frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}| i \in s_{t}^{(NC)}\right)\\&\quad +2Cov_{D}\left(\frac{{\hat{N}}_{t}^{(C,1)}}{E_{r}({\hat{N}}_{t}^{(C)} | s_{t}^{(C)})}\sum ^{}_{i \in s_{t}^{(C)}}\frac{E_{r}(R_{i,t}|s_{t}^{(C)})u_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}},\right. \\&\quad \qquad \left. \frac{{\hat{N}}_{t}^{(NC,1)}}{E_{r}\left({\hat{N}}_{t}^{(NC)^{*}}|s_{t}^{(NC)}\right)}\sum ^{}_{j \in s_{t}^{(NC)}}\frac{E_{r}(R_{j,t}|s_{t}^{(NC)})u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}}\right). \end{aligned}$$

Because \(E_{r}(\ \cdot \ |s_{t}^{(g)})\) is determined based on the response variable, the expected value is as follows.

$$\begin{aligned} \approx&V_{D}\left(\frac{{\hat{N}}_{t}^{(C,1)}}{E_{r}\left({\hat{N}}_{t}^{(C)}|s_{t}^{(C)}\right)} \sum ^{}_{s_{t}^{(C)}}\frac{E_{r}(R_{i,t}|s_{t}^{(C)})u_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}\right) \\ {}&+V_{D}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{E_{r}({\hat{N}}_{t}^{(NC)^{*}}|s_{t}^{(NC)})} \sum ^{}_{s_{t}^{(NC)}}\frac{E_{r}(R_{i,t}|s_{t}^{(NC)})u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}\right) \\ {}&+2\{E_{D}\left(\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C,1)}}\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\sum ^{}_{i \in s_{t}^{(C)}}\sum ^{}_{j \in s _{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}\frac{\phi _{j,t}^{(NC)}u_{j,t}}{\pi {j}\phi _{j,t}^{(NC)^{*}}}\right) \\ {}&\qquad -E_{D}\left(\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C,1)}}\sum ^{}_{i \in s_{t}^{(C)}}\frac{u_{i,t}}{\pi _{i}}\right)E_{D}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\sum ^{}_{j \in s_{t}^{(NC)}}\frac{\phi _{j,t}^{(NC)}u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}}\right)\}, \end{aligned}$$

where under Assumptions A.2 and A.5, \(E_{r}(R_{i,t}|s_{t}^{(C)})=\phi _{i,t}^{(C)}, \ E_{r}(R_{i,t}|s_{t}^{(NC)})=\phi _{i,t}^{(NC)},\)

\(E_{D}(\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C)}} \sum ^{}_{i \in s_{t}^{(C)}}\frac{u_{i,t}}{\pi _{i}})=0,\quad E_{r}({\hat{N}}_{t}^{(C)}|s_{t}^{(C)})={\hat{N}}_{t}^{(C,1)}\). Thus, we obtain the following expression.

$$\begin{aligned} \approx&V_{D}\left(\sum ^{}_{s_{t}^{(C)}}\frac{u_{i,t}}{\pi _{i}}\right) +V_{D}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\sum ^{}_{s_{t}^{(NC)}}\frac{\phi _{i,t}^{(NC)}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}})\\&+2E_{D}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\sum ^{}_{i \in s_{t}^{(C)}}\sum ^{}_{j \in s _{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}\frac{\phi _{j,t}^{(NC)}u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}}\right). \end{aligned}$$

Noting that \(V_{D}(I_{i}I_{j})=\Delta _{ij}(=\pi _{ij}-\pi _{i}\pi _{j}),\quad E_{D}(I_{i}I_{j})=\pi _{ij}\), we obtain the following.

$$\begin{aligned} \approx&\sum _{U_t^{(C)}}\sum _{U_t^{(C)}}\Delta _{ij}\frac{u_{i,t}}{\pi _{i}}\frac{u_{j,t}}{\pi _{j}} +(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}})^{2}\sum _{U_t^{(NC)}}\sum _{U_t^{(NC)}} \Delta _{ij}\frac{\phi _{i,t}^{(NC)}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}} \frac{\phi _{j,t}^{(NC)}u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}} \\&+2E_{D}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}})\sum ^{}_{i \in U_{t}^{(C)}}\sum ^{}_{j \in U_{t}^{(NC)}}\pi _{ij}(\frac{u_{i,t}}{\pi _{i}}\frac{\phi _{j,t}^{(NC)}u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}}). \end{aligned}$$

\(\square\)

(4) Proof of Theorem 4

We use the approximate Taylor linearization of Theorem (3) to induce the MSE in the group.

\(\textcircled {1}\) The \(E_{D}mse_{r}({\hat{t}}_{y_{t}^{(C)}}|s _{t}^{(C)})\) of the C group is given as follows.

$$\begin{aligned} E_{D}&mse_{r}({\hat{t}}_{y _{t}^{(C)}}|s_{t}^{(C)})\\&=E_{D}E_{r}\left({\hat{t}}_{y_{t}^{(C)}}-E_{r}({\hat{t}}_{y_{t}^{(C)}}) +E_{r}({\hat{t}}_{y_{t}^{(C)}})-t_{y_{t}}|s_{t}^{(C)}\right)^{2}\\&=E_{D}E_{r}\left(\sum ^{}_{s_{t}^{(C)}}\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}}^{(C)}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}u_{i,t} -E_{r}(\sum ^{}_{s_{t}^{(C)}}\frac{{\hat{N}}_{t}^{(C,1)}}{\hat{N_{t}}^{(C)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}u_{i,t})|s_{t}^{(C)}\right)^{2}\\&+E_{D}E_{r}\left[E_{r}(\sum ^{}_{s_{t}^{(C)}}\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}u_{i,t}) -\sum ^{}_{s_{t}^{(C)}}\frac{u_{i,t}}{\pi _{i}}|s_{t}^{(C)}\right]^{2}\\&=E_{D}V_{r}(\sum ^{}_{s_{t}^{(C)}}\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C)}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}|s_{t}^{(C)})\\&=E_{D}\left(\sum ^{}_{s_{t}^{(C)}}\phi _{i,t}^{(C)}(1-\phi _{i,t}^{(C)})(\frac{u _{i,t}}{\pi _{i}\phi _{i,t}^{(C)}})^{2}\right), \end{aligned}$$

where

$$\begin{aligned}&V_{r}(R_{i,t}|s_{t}^{(C)})=\Delta _{ij|s_{t}^{(C)}}=\phi _{i,t}^{(C)}(1-\phi _{i,t}^{(C)}),\quad E_{r}(R_{i,t}|s_{t}^{(C)})=\phi _{i,t}^{(C)},\\&E_{r}\left(\frac{{\hat{N}}_{t}^{(C,1)}}{{\hat{N}}_{t}^{(C)}}|s_{t}^{(C)}\right) \approx \frac{{\hat{N}}_{t}^{(C,1)}}{E _{r}({\hat{N}}_{t}^{(C)}|s_{t}^{(C)})} \approx 1. \end{aligned}$$

The second term bias is zero, leaving only the variance part of the C group. Hence, we derive the MSE of the NC group as follows.

\(\textcircled {2}\) \(E_{D}mse_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}|s_{t}^{(NC)})\) of the NC group

$$\begin{aligned} E_{D}&mse_{r}\left({\hat{t}}_{y_{t}^{(NC)^{*}}}|s_{t}^{(NC)}\right)\\&=E_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}-t_{y_{t}}|s_{t}^{(NC)})^{2}\\&=E_{D}E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}-E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}}) +E_{r}({\hat{t}}_{y_{t}^{(NC)^{*}}})-t_{y_{t}} |s_{t}^{(NC)})^{2}\\&=E_{D}E_{r}\left(\sum ^{}_{s_{t}^{(NC)}}\frac{{\hat{N}}_{t}^{(NC,1)}}{\hat{N_{t}}^{(NC)^{*}}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}u_{i,t} -E_{r}\left(\sum ^{}_{s_{t}^{(NC)}}\frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}u_{i,t}\right)|s_{t}^{(NC)}\right)^{2}\\&\quad +E_{D}E_{r}\left[E_{r}(\sum ^{}_{s_{t}^{(NC)}}\frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}u_{i,t}) -\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}|s_{t}^{(NC)}\right]^{2}\\&=E_{D}V_{r}\left(\sum ^{}_{s_{t}^{(NC)}}\frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)^{*}}}\frac{R_{i,t}u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}|s_{t}^{(NC)}\right)\\&\quad +E_{D}E_{r}\left[\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1)|s_{t}^{(NC)}\right]^{2}\\&=E_{D}\left(\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{\hat{N_{t}}^{(NC)^{*}}}\right)^{2} \sum ^{}_{s_{t}^{(NC)}}\Delta _{ij|s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}\frac{u_{j,t}}{\pi _{j}\phi _{j,t}^{(NC)^{*}}}\right)\\&\quad +E_{D}[\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1)]^{2}\\&=E_{D}\left(\sum ^{}_{s_{t}^{(NC)}}\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)})\left(\frac{{\hat{N}} _{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC)*}}\right)^{2}\left(\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}\right)^{2}\right)\\&\quad +E_{D}\left[\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1)\right]^{2}. \end{aligned}$$

Under Assumption A.2,   

$$\begin{aligned}&V_{r}(R_{i,t}|s_{t}^{(NC)})=\Delta _{ij|s_{t}^{(NC)}}=\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)}),\\&E_{r}(R_{i,t}|s_{t}^{(NC)})=\phi _{i,t}^{(NC)}. \end{aligned}$$

\(\square\)

(5) An asymptotic MSE for each estimator

The MSEs for each estimator are given as follows using the asymptotic MSE formula derived from Theorem 4.

\(\textcircled {1}\) The MSE of \({\hat{t}}_{y_{t}^{(NC)^{*}}}\) is given as follows.

$$\begin{aligned} E_{D}&mse_{r}\left({\hat{t}}_{y_{t}^{(NC)^{*}}}|s_{t}^{(NC)}\right)\\&\approx E_{D}(\sum ^{}_{s_{t}^{(NC)}}\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)})\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}}\right)^{2} \left(\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}})^{2}\right)\\&+E_{D}\left[\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}\left(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC)^{*}}} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(NC)^{*}}}-1\right)\right]^{2},\quad u_{i,t}=y_{i,t}-\tilde{y}_{t}^{(NC,1)}, \end{aligned}$$

where

$$\begin{aligned} N_{t}^{(NC)^{*}}&=E_{r}({\hat{N}}_{t}^{(NC)^{*}}|s_{t}^{(NC)})\\&=\sum ^{}_{s_{t}^{(NC)}}\frac{\phi _{i,t}^{(NC)}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}, \quad {\hat{N}}_{t}^{(NC)^{*}}=\sum ^{}_{s_{t}^{(NC)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)^{*}}}. \end{aligned}$$

\(\textcircled {2}\) The MSE of \({\hat{t}}_{y_{t}^{(NC)}}\) may be expressed as

$$\begin{aligned} E_{D}&mse_{r}({\hat{t}}_{y_{t}^{(NC)}}|s_{t}^{(NC)})\\&\approx E_{D}(\sum ^{}_{s_{t}^{(NC)}}\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)})(\frac{{\hat{N}}_{t}^{(NC,1)}}{{\hat{N}}_{t}^{(NC,1)}})^{2} (\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)}})^{2})\\&=E_{D}(\sum ^{}_{s_{t}^{(NC)}}\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)})(\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)}})^{2}),\quad u_{i,t}=y_{i,t}-\tilde{y}_{t}^{(NC,1)}, \end{aligned}$$

where

$$\begin{aligned} E_{r}({\hat{N}}_{t}^{(NC)}|s_{t}^{(NC)})&=E_{r}(\sum ^{}_{s_{t}^{(NC)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(NC)}}|s_{t}^{(NC)})\\&=\sum ^{}_{s_{t}^{(NC)}}\frac{1}{\pi _{i}}={\hat{N}}_{t}^{(NC,1)}. \end{aligned}$$

\(\textcircled {3}\) The MSE of \({\hat{t}}_{y_{t}^{(NC_{C})}}\) is given as

$$\begin{aligned} E_{D}&mse_{r}({\hat{t}}_{y_{t}^{(NC_{C})}}|s_{t}^{(NC)})\\&\approx E_{D}(\sum ^{}_{s_{t}^{(NC)}}\phi _{i,t}^{(NC)}(1-\phi _{i,t}^{(NC)}) (\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC_{C})}})^{2}(\frac{u_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}})^{2})\\&+E_{D}[\sum ^{}_{s_{t}^{(NC)}}\frac{u_{i,t}}{\pi _{i}}(\frac{{\hat{N}}_{t}^{(NC,1)}}{N_{t}^{(NC_{C})}} \frac{\phi _{i,t}^{(NC)}}{\phi _{i,t}^{(C)}}-1)]^{2},\quad u_{i,t}=y_{i,t}-\tilde{y}_{t}^{(NC,1)}, \end{aligned}$$

where

$$\begin{aligned} N_{t}^{(NC_{C})}&=E_{r}({\hat{N}}_{t}^{(NC_{C})}|s_{t}^{(NC)})\\&=\sum ^{}_{s_{t}^{(NC)}}\frac{\phi _{i,t}^{(NC)}}{\pi _{i}\phi _{i,t}^{(C)}}, \quad {\hat{N}}_{t}^{(NC_{C})}=\sum ^{}_{s_{t}^{(NC)}}\frac{R_{i,t}}{\pi _{i}\phi _{i,t}^{(C)}}. \end{aligned}$$

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, HA., Kim, YW. Nonresponse adjusted estimation based on a composite weighting method in a panel survey. J. Korean Stat. Soc. 52, 304–329 (2023). https://doi.org/10.1007/s42952-023-00202-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42952-023-00202-x

Keywords

Navigation