Abstract
Planned Missingness (PM) designs, in which researchers deliberately collect only partial data, have enjoyed a recent growth in popularity. Among other benefits these designs have been proven capable of reducing the study costs and alleviating participant burden. Past research has shown that Split Form PM designs can be effective in simplifying complex surveys while Wave Missingness PM designs act similarly for Longitudinal studies. However, less work has been done to inform how to implement PM structures into studies which incorporate elements of both survey and longitudinal designs. Specifically, in studies where a questionnaire is given to participants at multiple measurement occasions the best way to design missingness is still unclear. To address this deficiency, data in this hybrid format was simulated under both Split Form and Wave Missingness PM structures. Multiple imputation techniques were applied to estimate a multilevel logistic model in each of the simulations. Estimated parameters were compared to the true values to see which PM design allowed us to best capture the true model. The results of this study indicate that, compared to the Split Form Design, the Wave Missingness design consistently performed less effectively in capturing the multilevel model. Thus, in the context of longitudinal surveys this study recommends the use of Split Form missingness designs, which performs well under a number of different conditions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andrés Houseman, E. and Milton, D.K. (2006). Partial questionnaire designs, questionnaire non-response, and attributable fraction: applications to adult onset asthma. Stat. Med.25, 9, 1499–1519.
Collins, L.M., Schafer, J.L. and Kam, C.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol. Methods6, 4, 330.
Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Methodol.39, 1, 1–38.
Duncan, Terry E, Duncan, Susan C and Strycker, L.A. (2013). An introduction to latent variable growth curve modeling: Concepts, issues, and application. Routledge Academic, Abingdon.
Gottschall, A.C., West, S.G. and Enders, C.K. (2012). A comparison of item-level and scale-level multiple imputation for questionnaire batteries. Multivar. Behav. Res.47, 1, 1–25.
Graham, J.W, Hofer, S.M. and MacKinnon, D.P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns An application of maximum likelihood procedures. Multivar. Behav. Res.31, 2, 197–218.
Graham, J.W., Taylor, B.J. and Cumsille, P.E. (2001). Planned missing-data designs in analysis of change.
Graham, J.W., Taylor, B.J., Olchowski, A.E. and Cumsille, P.E. (2006). Planned missing data designs in psychological research. Psychol. Methods11, 4, 323.
Graham, J.W., Olchowski, A.E. and Gilreath, T.D. (2007). How many imputations are really needed? some practical clarifications of multiple imputation theory. Prev. Sci.8, 3, 206–213.
Harel, O. (2007). Inferences on missing information under multiple imputation and two-stage multiple imputation. Stat. Methodol.4, 1, 75–89.
Harel, O. and Zhou, X.-H. (2007). Multiple imputation: review of theory, implementation and software. Stat. Med.26, 16, 3057–3077.
Harel, O., Stratton, J. and Aseltine, R. (2015). Designed missingness to better estimate efficacy of behavioral studies-application to suicide prevention trials. J. Med. Stat. Inf.3, 1, 2.
Kalichman, S.C., Amaral, C.M., White, D., Swetsze, C., Pope, H., Kalichman, M.O., Cherry, C. and Eaton, L. (2009). Prevalence and clinical implications of interactive toxicity beliefs regarding mixing alcohol and antiretroviral therapies among people living with hiv/aids. AIDS patient care and STDs23, 6, 449–454.
Kalichman, S.C., Amaral, C.M., White, D., Swetsze, C., Kalichman, M.O, Cherry, C. and Eaton, L. (2012). Alcohol and adherence to antiretroviral medications: interactive toxicity beliefs among people living with hiv. J. Assoc. Nurses AIDS Care23, 6, 511–520.
King, G. and Zeng, L. (2001). Logistic regression in rare events data. Policy Anal.9, 2, 137–163.
Little, R.J.A. and Rubin, D.B. (2014). Statistical analysis with missing data. Wiley, New York.
Little, T.D. (2013). Longitudinal structural equation modeling. Guilford Press, New York.
Little, T.D. and Rhemtulla, M. (2013). Planned missing data designs for developmental researchers. Child Dev. Perspect.7, 4, 199–204.
McArdle, J.J. and Woodcock, R.W. (1997). Expanding test–retest designs to include developmental time-lag components. Psychol. Methods2, 4, 403.
Pellowski, J.A., Kalichman, S.C., Kalichman, M.O. and Cherry, C. (2016). Alcohol-antiretroviral therapy interactive toxicity beliefs and daily medication adherence and alcohol use among people living with hiv. AIDS care28, 8, 963–970.
Raghunathan, T.E. (2015). Missing data analysis in practice. CRC Press, Boca Raton.
Raghunathan, T.E. and Grizzle, J.E. (1995). A split questionnaire survey design. J. Am. Stat. Assoc.90, 429, 54–63.
Rhemtulla, M., Jia, F., Wei, W.U. and Little, T.D. (2014). Planned missing designs to optimize the efficiency of latent growth parameter estimates. International Journal of Behavioral Development, 0165025413514324.
Rhemtulla, M., Savalei, V. and Little, T.D. (2016). On the asymptotic relative efficiency of planned missingness designs. Psychometrika81, 1, 60–89.
Rubin, D.B. (2004). Multiple imputation for nonresponse in surveys. Wiley classics library. Wiley, New York. ISBN 9780471655749. https://books.google.com/books?id=bQBtw6rx_mUC.
Rubin, D.B. (1976). Inference and missing data. Biometrika63, 3, 581–592.
Saris, W.E., Satorra, A. and Germà, C. (2004). 8. a new approach to evaluating the quality of measurement instruments: The split-ballot MTMM design. Sociol. Methodol.34, 1, 311–347.
Shoemaker, D.M. (1973). Principles and procedures of multiple matrix sampling, Ballinger.
Silvia, P.J., Kwapil, T.R., Walsh, M.A. and Myin-Germeys, I. (2014). Planned missing-data designs in experience-sampling research Monte carlo simulations of efficient designs for assessing within-person constructs. Behav. Res. Methods46, 1, 41–54.
Sirotnik, K. and Wellington, R. (1977). Incidence sampling: An integrated theory for “matrix sampling”. J. Educ. Meas.14, 4, 343–399.
Thomas, N., Raghunathan, T.E., Schenker, N., Katzoff, M.J. and Johnson, C.L. (2006). An evaluation of matrix sampling methods using data from the national health and nutrition examination survey. Survey Methodol.32, 2, 217.
van Buuren, S. and Groothuis-Oudshoorn, K. (2011). Mice: Multivariate imputation by chained equations in r. J. Stat. Softw.45, 3, 1–67. http://www.jstatsoft.org/v45/i03/.
Wacholder, S., Carroll, R.J., Pee, D. and Gail, M.H. (1994). The partial questionnaire design for case-control studies. Stat. Med.13, 5-7, 623–634.
Zeger, L.M. and Thomas, N. (1997). Efficient matrix sampling instruments for correlated latent traits: examples from the national assessment of educational progress. J. Am. Stat. Assoc.92, 438, 416–425.
Acknowledgements
The first version of this manuscript was based on the undergraduate research of Nicholas Illenberger.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wood, J., Matthews, G.J., Pellowski, J. et al. Comparing Different Planned Missingness Designs in Longitudinal Studies. Sankhya B 81, 226–250 (2019). https://doi.org/10.1007/s13571-018-0170-5
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13571-018-0170-5