Skip to main content
SpringerLink
Log in

Multiple imputation of cognitive performance as a repeatedly measured outcome

  • METHODS
  • Published:
European Journal of Epidemiology Aims and scope Submit manuscript

Abstract

Longitudinal studies of cognitive performance are sensitive to dropout, as participants experiencing cognitive deficits are less likely to attend study visits, which may bias estimated associations between exposures of interest and cognitive decline. Multiple imputation is a powerful tool for handling missing data, however its use for missing cognitive outcome measures in longitudinal analyses remains limited. We use multiple imputation by chained equations (MICE) to impute cognitive performance scores of participants who did not attend the 2011–2013 exam of the Atherosclerosis Risk in Communities Study. We examined the validity of imputed scores using observed and simulated data under varying assumptions. We examined differences in the estimated association between diabetes at baseline and 20-year cognitive decline with and without imputed values. Lastly, we discuss how different analytic methods (mixed models and models fit using generalized estimate equations) and choice of for whom to impute result in different estimands. Validation using observed data showed MICE produced unbiased imputations. Simulations showed a substantial reduction in the bias of the 20-year association between diabetes and cognitive decline comparing MICE (3–4 % bias) to analyses of available data only (16–23 % bias) in a construct where missingness was strongly informative but realistic. Associations between diabetes and 20-year cognitive decline were substantially stronger with MICE than in available-case analyses. Our study suggests when informative data are available for non-examined participants, MICE can be an effective tool for imputing cognitive performance and improving assessment of cognitive decline, though careful thought should be given to target imputation population and analytic model chosen, as they may yield different estimands.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychol Methods. 2002;7(2):147–177. http://www.ncbi.nlm.nih.gov/pubmed/12090408. Accessed 22 Mar 2015.

  2. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377–99. doi:10.1002/sim.4067.

    Article  PubMed  Google Scholar 

  3. Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59(10):1087–91. doi:10.1016/j.jclinepi.2006.01.014.

    Article  PubMed  Google Scholar 

  4. Rubin DB. Multiple imputation for nonresponse surveys. New York: Wiley; 1987.

    Book  Google Scholar 

  5. Seaman SR, Bartlett JW, White IR. Multiple imputation of missing covariates with non-linear effects and interactions: an evaluation of statistical methods. BMC Med Res Methodol. 2012;12(1):46. doi:10.1186/1471-2288-12-46.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Klebanoff MA, Cole SR. Use of multiple imputation in the epidemiologic literature. Am J Epidemiol. 2008;168(4):355–7. doi:10.1093/aje/kwn071.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Laird NM. Missing data in longitudinal studies. Stat Med. 1988;7(1–2):305–15. doi:10.1002/sim.4780070131.

    Article  CAS  PubMed  Google Scholar 

  8. Gad AM, Darwish NMM. A shared parameter model for longitudinal data with missing values. Am J Appl Math Stat. 2013;1(2):30–5. doi:10.12691/ajams-1-2-3.

    Article  Google Scholar 

  9. Creemers A, Hens N, Aerts M, Molenberghs G, Verbeke G, Kenward MG. Generalized shared-parameter models and missingness at random. Stat Modelling. 2011;11(4):279–310. doi:10.1177/1471082X1001100401.

    Article  Google Scholar 

  10. Weuve J, Tchetgen Tchetgen EJ, Glymour MM, et al. Accounting for bias due to selective attrition: the example of smoking and cognitive decline. Epidemiology. 2012;23(1):119–28. doi:10.1097/EDE.0b013e318230e861.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Robins JM, Hernán MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11(5):550–560. http://www.ncbi.nlm.nih.gov/pubmed/10955408. Accessed 25 Aug 2015.

  12. Rabbitt P, Diggle P, Holland F, McInnes L. Practice and drop-out effects during a 17-year longitudinal study of cognitive aging. J Gerontol B Psychol Sci Soc Sci. 2004;59(2):P84–P97. http://www.ncbi.nlm.nih.gov/pubmed/15014091. Accessed 13 Aug 2015.

  13. Shen C, Gao S. A mixed-effects model for cognitive decline with non-monotone non-response from a two-phase longitudinal study of dementia. Stat Med. 2007;26(2):409–25. doi:10.1002/sim.2454.

    Article  PubMed  Google Scholar 

  14. Gard T, Hölzel BK, Lazar SW. The potential effects of meditation on age-related cognitive decline: a systematic review. Ann N Y Acad Sci. 2014;1307:89–103. doi:10.1111/nyas.12348.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Gerstorf D, Herlitz A, Smith J. Stability of sex differences in cognition in advanced old age: the role of education and attrition. J Gerontol B Psychol Sci Soc Sci. 2006;61(4):P245–P249. http://www.ncbi.nlm.nih.gov/pubmed/16855037. Accessed 13 Aug 2015.

  16. Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res. 2011;20(1):40–9. doi:10.1002/mpr.329.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Jones M, Mishra GD, Dobson A. Analytical results in longitudinal studies depended on target of inference and assumed mechanism of attrition. J Clin Epidemiol. 2015;68(10):1165–75. doi:10.1016/j.jclinepi.2015.03.011.

    Article  PubMed  Google Scholar 

  18. The Atherosclerosis Risk in Communities (ARIC) Study: design and objectives. The ARIC investigators. Am J Epidemiol. 1989;129(4):687–702. http://www.ncbi.nlm.nih.gov/pubmed/2646917.

  19. Knopman DS, Ryberg S. A verbal memory test with high predictive accuracy for dementia of the Alzheimer type. Arch Neurol. 1989;46(2):141–145. http://www.ncbi.nlm.nih.gov/pubmed/2916953.

  20. Wechsler D. Manual for the wechsler adult intelligence scale, Revised. 1981.

  21. Benton A, Hamsher K. Multilingual aphasia examination. 2nd ed. Oowa City: AJA Associates; 1989.

    Google Scholar 

  22. Brandt J, Spencer M, Folstein M. The telephone interview for cognitive status. Neuropsychiatry Neuropsychol Behav Neurol. 1988;1(2):111–8.

    Google Scholar 

  23. Knopman DS, Roberts RO, Geda YE, et al. Validation of the telephone interview for cognitive status-modified in subjects with normal cognition, mild cognitive impairment, or dementia. Neuroepidemiology. 2010;34(1):34–42. doi:10.1159/000255464.

    Article  PubMed  Google Scholar 

  24. Plassman BLPD, Newman TTBS, Welsh KAPD, Helms MBS, Breitner JCS. Properties of the telephone interview for cognitive status: application in epidemiological and longitudinal studies. Neuropsychiatry Neuropsychol Behav Neurol. 1994;7(3):235–41.

    Google Scholar 

  25. Manual 19. Surveillance of Dementia in the ARIC Cohort.; 2015.

  26. Kurland BF, Johnson LL, Egleston BL, Diehr PH. Longitudinal data with follow-up truncated by death: match the analysis method to research aims. Stat Sci. 2009;24(2):211. doi:10.1214/09-STS293.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Rawlings AM, Sharrett AR, Schneider ALC, et al. Diabetes in midlife and cognitive change over 20 years: a cohort study. Ann Intern Med. 2014;161(11):785–93. doi:10.7326/M14-0737.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Little RJA, Rubin DB. Statistical analysis with missing data. 2nd ed. Hoboken: Wiley; 2002.

    Google Scholar 

  29. Little RJA. Modeling the drop-out mechanism in repeated-measures studies. J Am Stat Assoc. 1995;90(431):1112. doi:10.2307/2291350.

    Article  Google Scholar 

  30. Van Buuren S, Brand JPL, Groothuis-Oudshoorn CGM, Rubin DB. Fully conditional specification in multivariate imputation. J Stat Comput Simul. 2006;76(12):1049–64. doi:10.1080/10629360600810434.

    Article  Google Scholar 

  31. Bartlett JW, Seaman SR, White IR, Carpenter JR. Multiple imputation of covariates by fully conditional specification: accommodating the substantive model. Stat Methods Med Res. 2014;. doi:10.1177/0962280214521348.

    PubMed  Google Scholar 

  32. Manton KG, Gu X, Lamb VL. Long-term trends in life expectancy and active life expectancy in the United States. Popul Dev Rev. 2006;32(1):81–105. doi:10.1111/j.1728-4457.2006.00106.x.

    Article  Google Scholar 

  33. Sauvaget C, Tsuji I, Haan MN, Hisamichi S. Trends in dementia-free life expectancy among elderly members of a large health maintenance organization. Int J Epidemiol. 1999;28(6):1110–1118. http://www.ncbi.nlm.nih.gov/pubmed/10661655. Accessed 27 June 2016.

  34. Chaix B, Evans D, Merlo J, Suzuki E. Commentary: weighing up the dead and missing: reflections on inverse-probability weighting and principal stratification to address truncation by death. Epidemiology. 2012;23(1):129–131; discussion 132–137. doi:10.1097/EDE.0b013e3182319159.

  35. Stuart EA, Azur M, Frangakis C, Leaf P. Multiple imputation with large data sets: a case study of the Children’s Mental Health Initiative. Am J Epidemiol. 2009;169(9):1133–1139. doi:10.1093/aje/kwp026.

  36. Power MC, Tchetgen EJ, Sparrow D, Schwartz J, Weisskopf MG. Blood pressure and cognition: factors that may account for their inconsistent association. Epidemiology. 2013;24(6):886–93. doi:10.1097/EDE.0b013e3182a7121c.

    Article  PubMed  Google Scholar 

  37. Seaman SR, White IR, Copas AJ, Li L. Combining multiple imputation and inverse-probability weighting. Biometrics. 2012;68(1):129–37. doi:10.1111/j.1541-0420.2011.01666.x.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Han P. Combining inverse probability weighting and multiple imputation to improve robustness of estimation. Scand J Stat. 2015;. doi:10.1111/sjos.12177.

    Google Scholar 

  39. Brand JPL. Development, implementation and evaluation of multiple imputation strategies for the statistical analysis of incomplete data sets. 1999.

  40. Scharfstein DO, Irizarry RA. Generalized additive selection models for the analysis of studies with potentially nonignorable missing outcome data. Biometrics. 2003;59(3):601–613. http://www.ncbi.nlm.nih.gov/pubmed/14601761. Accessed 23 Aug 2015.

  41. Greenland S. Basic methods for sensitivity analysis of biases. Int J Epidemiol. 1996;25(6):1107–1116. http://www.ncbi.nlm.nih.gov/pubmed/9027513. Accessed 23 Aug 2015.

Download references

Acknowledgments

The Atherosclerosis Risk in Communities Study is carried out as a collaborative study supported by National Heart, Lung, and Blood Institute contracts (HHSN268201100005C, HHSN268201100006C, HHSN268201100007C, HHSN268201100008C, HHSN268201100009C, HHSN268201100010C, HHSN268201100011C, and HHSN268201100012C, with the ARIC carotid MRI examination funded by U01HL075572-01). Neurocognitive data is collected by U01 HL096812, HL096814, HL096899, HL096902, HL096917 from the NHLBI and the National Institute of Neurological Disorders and Stroke, and with previous brain MRI examinations funded by R01-HL70825 from the NHLBI. The authors thank the staff and participants of the ARIC study for their important contributions.

Author information

Authors and Affiliations

  1. Department of Epidemiology, Welch Center for Prevention, Epidemiology and Clinical Research, Johns Hopkins Bloomberg School of Public Health, 2024 E. Monument Street, Suite 2-600, Baltimore, MD, 21287, USA

    Andreea Monica Rawlings, Yingying Sang, Albert Richey Sharrett, Josef Coresh, Alden Lawrence Gross, Jennifer Anne Deal & Melinda Carolyn Power

  2. Center of Biostatistics and Bioinformatics, University of Mississippi Medical Center, Jackson, MS, USA

    Michael Griswold

  3. Department of Epidemiology, University of North Carolina, Chapel Hill, NC, USA

    Anna Maria Kucharska-Newton & Priya Palta

  4. Department of Biostatistics, University of North Carolina, Chapel Hill, NC, USA

    Lisa Miller Wruck

  5. Department of Epidemiology and Biostatistics, George Washington University Milken Institute of Public Health, Washington, DC, USA

    Melinda Carolyn Power

  6. Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA

    Karen Jean Bandeen-Roche

Authors
  1. Andreea Monica Rawlings
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingying Sang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Albert Richey Sharrett
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Josef Coresh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michael Griswold
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anna Maria Kucharska-Newton
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Priya Palta
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Lisa Miller Wruck
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Alden Lawrence Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Jennifer Anne Deal
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Melinda Carolyn Power
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Karen Jean Bandeen-Roche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andreea Monica Rawlings.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 893 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rawlings, A.M., Sang, Y., Sharrett, A.R. et al. Multiple imputation of cognitive performance as a repeatedly measured outcome. Eur J Epidemiol 32, 55–66 (2017). https://doi.org/10.1007/s10654-016-0197-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10654-016-0197-8

Keywords

Search

Navigation