Abstract
The study of adolescent development rests on methodologically appropriate collection and interpretation of longitudinal data. While all longitudinal studies of adolescent development involve missing data, the methods to treat missingness that have been recommended most often focus on missing data from cross-sectional studies. The problems of missing data in longitudinal studies are not described well, there are not many statistical software programs developed for researchers to use, and there are no longitudinal empirical examples involving adolescent development that show the extent to which different missing data procedures can yield different results. Data from the first three waves of the 4-H Study of Positive Youth Development were used to provide such an illustration. The sample contains 2,265 participants (56.7% females) who were in Grade 5 at Wave 1, in Grade 6 at Wave 2, and in Grade 7 at Wave 3, and varied in race, ethnicity, socioeconomic status, family structure, rural–urban location, and geographic region. The results showed that three missing data techniques, i.e., listwise deletion, direct maximum likelihood (DirML), and multiple imputation (MI), did not yield comparable results for research questions assessing different aspects of development (i.e., change over time or prediction effects). The results indicated also that listwise deletion should not be used. Instead, both DirML and MI methods should be used to determine if and how results change when these procedures are employed.
This is a preview of subscription content, access via your institution.
References
Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage.
Arbuckle, J. L. (1996). Full information estimation procedure in the presence of incomplete data. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advance structural equation modeling (pp. 243–277). Mahwah, NJ: Lawrence Erlbaum.
Armsden, G., & Greenberger, M. (1987). The inventory of parent and peer attachment: Individual differences and their relationship to psychological well-being in adolescence. Journal of Youth and Adolescence, 16, 427–452.
Baltes, P. B., Reese, H. W., & Nesselrode, J. R. (1988). Introduction to research methods: Life-span developmental psychology. Hillsdale, NJ: Lawrence Erlbaum.
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multivariate regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
Collins, L. M., Schafer, J. W., & Kam, C. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351.
Daniels, M. J., & Hogan, J. W. (2000). Reparameteirizing the pattern mixture model for sensitivity analysis under informative dropout. Biometrics, 56, 1241–1248.
Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8(3), 430–457.
Foster, E. M., Fang, G. Y., & Conduct Problems Prevention Research Group. (2004). Alternative methods for handling attrition in prevention research: An illustration using data from Fast Track. Evaluation Review, 28, 434–464.
Graham, J. W., Cumsille, P. E., & Elek-Fisk, E. (2003). Methods for handling missing data. In J. A. Schinka & W. F. Velicer (Eds.), Research methods in psychology (pp. 87–114). Vol. 2 of Handbook of psychology (I. B. Weiner, Editor-in-Chief). New York: Wiley.
Graham, J. W., & Donaldson, S. I. (1993). Evaluating interventions with differential attrition: The importance of nonresponse mechanisms and use of follow-up data. Journal of Applied Psychology, 78, 119–128.
Graham, J. W., & Hofer, S. M. (2000). Multiple imputation in multivariate research. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 201–218). Mahwah, NJ: Lawrence Erlbaum.
Graham, J. W., Hofer, S. M., & MacKinnon, D. P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31, 197–218.
Graham, J. W., Olchowski, A. E., & Gilreath, T. D. (2007). How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science, 8, 206–213.
Gray, M. R., & Steinberg, L. (1999). Unpacking authoritative parenting: Reassessing a multidimensional construct. Journal of Marriage and the Family, 61, 574–587.
Harel, O., & Zhou, X. H. (2007). Multiple imputation: Review of theory, implementation and software. Statistics in Medicine, 26, 3057–3077.
Harter, S. (1982). The perceived competence scale for children. Child Development, 53, 87–98.
Harter, S. (1983). Supplementary description of the self-perception profile for children: Revision of the perceived competence scale for children. Unpublished manuscript, University of Denver.
Jelicic, H., Phelps, E., & Lerner, R. M. (2009). Use of missing data methods in longitudinal studies: The persistence of bad practices in developmental psychology. Developmental Psychology, 45(4), 1195–1199.
Jones, M. P. (1996). Indicator and stratification methods for missing explanatory variables in multiple linear regression. Journal of the American Statistical Association, 91(433), 222–230.
Jöreskog, K. G., & Sörbom, D. (1996). LISREL 8: User’s reference guide. Chicago: Scientific Software International.
Lerner, R. M., Lerner, J. V., Almerigi, J., Theokas, C., Phelps, E., Gestsdottir, S., et al. (2005). Positive youth development, participation in community youth development programs, and community contributions of fifth grade adolescents: Findings from the first wave of the 4-H Study of Positive Youth Development. Journal of Early Adolescence, 25(1), 17–71.
Lerner, R. M., & Steinberg, L. (Eds.). (2009). Handbook of adolescent psychology (3rd ed.). New York: Wiley.
Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data (2nd ed.). Hoboken, NJ: Wiley.
Molenberghs, G., Kenward, M. G., & Lesaffre, E. (1997). The analysis of longitudinal ordinal data with non-random dropout. Biometrika, 84, 33–44.
Peugh, J. L., & Enders, C. K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74(4), 525–556.
Phelps, E., Balsano, A. B., Fay, K., Peltz, J. S., Zimmerman, S., Lerner, R. M., et al. (2007). Nuances in early adolescent developmental trajectories of positive and problematic/risk behaviors: Findings from the 4-H study of positive youth development. Child and Adolescent Psychiatric Clinics of North America, 16(2), 473–496.
Phelps, E., Zimmerman, S., Warren, A. E. A., Jelicic, H., von Eye, A., & Lerner, R. M. (2009). The structure and developmental course of Positive Youth Development (PYD) in early adolescence: Implications for theory and practice. Journal of Applied Developmental Psychology, 30(5), 571–584.
Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401.
Roth, P. L., Switzer, F. S., & Switzer, D. M. (1999). Missing data in multiple item scales: Monte Carlo analysis of missing data techniques. Organizational Research Methods, 2(3), 211–232.
Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3), 581–592.
Schafer, J. L. (1997). Analysis of incomplete multivariate data. London, UK: Chapman & Hall.
Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.
Schafer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst’s perspective. Multivariate Behavioral Research, 33(4), 545–571.
Schludermann, E., & Schluderman, S. (1970). Replicability of factors in children’s reports of parent behavior (CRPBI). Journal of Psychology, 76, 239–249.
Sinharay, S., Stern, H. S., & Russell, D. (2001). The use of multiple imputation for the analysis of missing data. Psychological Methods, 6(4), 317–329.
Small, S. A., & Kerns, D. (1993). Unwanted sexual activity among peers during early and middle adolescence: Incidence and risk factors. Journal of Marriage and the Family, 55, 941–952.
von Eye, A. (Ed.). (1990a). Statistical methods in longitudinal research. Vol. 1. Principles and structuring change. New York: Academic Press.
von Eye, A. (Ed.). (1990b). Statistical methods in longitudinal research. Vol. 2. Time series and categorical longitudinal data. New York: Academic Press.
Wothke, W. (2000). Longitudinal and multigroup modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 219–240). Mahwah, NJ: Lawrence Erlbaum.
Acknowledgments
This research was supported in part by a grant to Richard M. Lerner by the National 4-H Council. The authors thank Aline Sayer and Avron Spiro for their valuable comments. This article is based in part on a dissertation submitted by the first author in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Child Development at Tufts University.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jeličić, H., Phelps, E. & Lerner, R.M. Why Missing Data Matter in the Longitudinal Study of Adolescent Development: Using the 4-H Study to Understand the Uses of Different Missing Data Methods. J Youth Adolescence 39, 816–835 (2010). https://doi.org/10.1007/s10964-010-9542-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10964-010-9542-5