Neuropsychology Review

, Volume 18, Issue 3, pp 194–213 | Cite as

Methodological Challenges in Causal Research on Racial and Ethnic Patterns of Cognitive Trajectories: Measurement, Selection, and Bias

  • M. Maria GlymourEmail author
  • Jennifer Weuve
  • Jarvis T. Chen


Research focused on understanding how and why cognitive trajectories differ across racial and ethnic groups can be compromised by several possible methodological challenges. These difficulties are especially relevant in research on racial and ethnic disparities and neuropsychological outcomes because of the particular influence of selection and measurement in these contexts. In this article, we review the counterfactual framework for thinking about causal effects versus statistical associations. We emphasize that causal inferences are key to predicting the likely consequences of possible interventions, for example in clinical settings. We summarize a number of common biases that can obscure causal relationships, including confounding, measurement ceilings/floors, baseline adjustment bias, practice or retest effects, differential measurement error, conditioning on common effects in direct and indirect effects decompositions, and differential survival. For each, we describe how to recognize when such biases may be relevant and some possible analytic or design approaches to remediating these biases.


Causal research Racial and ethnic disparities Cognitive trajectory Neuropsychological research Counterfactuals Directed acyclic graphs Measurement error Selection 



Maria Glymour was a Robert Wood Johnson Health and Society Scholar at Columbia University when this was written. The authors gratefully acknowledge funding support from the National Institutes of Health (ES005257) and a Robert Wood Johnson Health and Society Program seed grant.


  1. Angrist, J. D., Imbens, G. W., et al. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434), 444–455.CrossRefGoogle Scholar
  2. Angrist, J. D., & Krueger, A. B. (2001). Instrumental variables and the search for identification: from supply and demand to natural experiments. Journal of Economic Perspectives, 15(4), 69–85.Google Scholar
  3. Anonymous (1998). Coronary heart disease mortality trends among whites and blacks Appalachia and United States, 1980–1993. MMWR CDC Surveillance Summaries, 47(46), 1005–1008.Google Scholar
  4. Barker, D. J. (1990). The fetal and infant origins of adult disease. BMJ, 301(6761), 1111.PubMedGoogle Scholar
  5. Baron, R. M., & Kenny, D. A. (1986). The moderator mediator variable distinction in social psychological-research—Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182.PubMedCrossRefGoogle Scholar
  6. Berkman, L. F. (1986). The association between educational attainment and mental status examinations: Of etiologic significance for senile dementias or not. Journal of Chronic Diseases, 39(3), 171–175.PubMedCrossRefGoogle Scholar
  7. Blakely, T. A. (2002). Commentary: Estimating direct and indirect effects—Fallible in theory, but in the real world. International Journal of Epidemiology, 31, 166–167.PubMedCrossRefGoogle Scholar
  8. Carpenter, J., & Bithell, J. (2000). Bootstrap confidence intervals: When, which, what? A practical guide for medical statisticians. Statistics in Medicine, 19, 1141–1164.PubMedCrossRefGoogle Scholar
  9. Catalano, R., Bruckner, T., et al. (2008). Ambient temperature predicts sex ratios and male longevity. Proceedings of the National Academy of Sciences of the United States of America, 105(6), 2244–2247.PubMedCrossRefGoogle Scholar
  10. CDC (1998). Trends in ischemic heart disease deaths rates for blacks and whites—United States, 1981–1995. MMWR CDC Surveillance Summaries, 47(44), 945–949.Google Scholar
  11. Chay, K. Y., & Powell, J. L. (2001). Semiparametric censored regression models. Journal of Economic Perspectives, 15(4), 29–42.CrossRefGoogle Scholar
  12. Clogg, C. C., Petkova, E., et al. (1995). Statistical methods for comparing regression coefficients between models. American Journal of Sociology, 100(5), 1261.CrossRefGoogle Scholar
  13. Cole, S. R., & Hernán, M. A. (2002). Fallibility in estimating direct effects. International Journal of Epidemiology, 31(1), 163–165.PubMedCrossRefGoogle Scholar
  14. Cole, S. R., Hernan, M. A., et al. (2005). Marginal structural models for estimating the effect of highly active antiretroviral therapy initiation on CD4 cell count. American Journal of Epidemiology, 162(5), 471–478.PubMedCrossRefGoogle Scholar
  15. Collie, A., Maruff, P., et al. (2003). The effects of practice on the cognitive test performance of neurologically normal individuals assessed at brief test–retest intervals. Journal of the International Neuropsychological Society, 9(3), 419–428.PubMedCrossRefGoogle Scholar
  16. Corti, M. C., Guralnik, J. M., et al. (1999). Evidence for a Black–White crossover in all-cause and coronary heart disease mortality in an older population: The North Carolina EPESE. American Journal of Public Health, 89(3), 308–314.PubMedGoogle Scholar
  17. Currie, J. (1995). Welfare and the well-being of children. Chur Switzerland: Harwood Academic.Google Scholar
  18. Didelez, V., & Sheehan, N. (2007). Mendelian randomization as an instrumental variable approach to causal inference. Statistical Methods in Medical Research, 16(4), 309–330.PubMedCrossRefGoogle Scholar
  19. Elo, I. T., & Preston, S. H. (1994). Estimating African–American mortality from inaccurate data. Demography, 31(3), 427–458.PubMedCrossRefGoogle Scholar
  20. Glymour, M. M. (2006a). Natural experiments and instrumental variables analyses in social epidemiology. In J. M. Oakes, & J. S. Kaufman (Eds.), Methods in social epidemiology. San Francisco: Jossey-Bass.Google Scholar
  21. Glymour, M. M. (2006b). Using causal diagrams to understand common problems in social epidemiology. In J. M. Oakes, & J. S. Kaufman (Eds.), Methods in social epidemiology. San Francisco: Jossey-Bass.Google Scholar
  22. Glymour, M. M., & Greenland, S. (2008). Causal diagrams. In K. J. Rothman, S. Greenland, & T. L. Lash (Eds.), Modern epidemiology (pp. 183–210). Philadelphia: Lippincott Williams & Wilkins.Google Scholar
  23. Glymour, M. M., Kawachi, I., et al. (2008). Does childhood schooling affect old age memory or mental status? Using state schooling laws as natural experiments. Journal of Epidemiology and Community Health, 62, 532–537.PubMedCrossRefGoogle Scholar
  24. Glymour, M. M., Weuve, J., et al. (2005). When is baseline adjustment useful in analyses of change? An example with education and cognitive change. American Journal of Epidemiology, 162(3), 267–278.PubMedCrossRefGoogle Scholar
  25. Greene, W. H. (2000). Econometric analysis. Upper Saddle River: Prentice-Hall.Google Scholar
  26. Greenland, S. (2000). An introduction to instrumental variables for epidemiologists. International Journal of Epidemiology, 29(4), 722–729.PubMedCrossRefGoogle Scholar
  27. Greenland, S. (2003). Quantifying biases in causal models: Classical confounding vs collider-stratification bias. Epidemiology, 14(3), 300–306.PubMedCrossRefGoogle Scholar
  28. Greenland, S., Pearl, J., et al. (1999). Causal diagrams for epidemiologic research. Epidemiology, 10(1), 37–48.PubMedCrossRefGoogle Scholar
  29. Hernán, M. A., Alonso, A., et al. (2008). Cigarette smoking and dementia: potential selection bias in the elderly. Epidemiology, 19(3), 448.PubMedCrossRefGoogle Scholar
  30. Hernán, M. A., Brumback, B., et al. (2000). Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology, 11(5), 561–570.PubMedCrossRefGoogle Scholar
  31. Hernan, M. A., Robins, J. M., et al. (2005). Statistical issues arising in the Women’s Health Initiative—Discussion. Biometrics, 61(4), 922–930.CrossRefGoogle Scholar
  32. Hines, L. M., Stampfer, M. J., et al. (2001). Genetic variation in alcohol dehydrogenase and the beneficial effect of moderate alcohol consumption on myocardial infarction. New England Journal of Medicine, 344(8), 549–555.PubMedCrossRefGoogle Scholar
  33. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81(396), 945–960.CrossRefGoogle Scholar
  34. Horton, A. M. (1992). Neurospsychological practice effects * age: A brief note. Perceptual & Motor Skills, 75(1), 257–258.CrossRefGoogle Scholar
  35. Howard, G., & Goff, D. C. (1998). A call for caution in the interpretation of the observed smaller relative importance of risk factors in the elderly. Annals of Epidemiology, 8(7), 411–414.PubMedCrossRefGoogle Scholar
  36. Judd, C. M., & Kenny, D. A. (1981). Process analysis—estimating mediation in treatment evaluations. Evaluation Review, 5(5), 602–619.CrossRefGoogle Scholar
  37. Kalbfleisch, J. D., & Prentice, R. L. (1980). The statistical analysis of failure time data. New York: Wiley.Google Scholar
  38. Kaplan, G. A., Haan, M. N., et al. (1999). Understanding changing risk factor associations with increasing age in adults. Annual Review of Public Health, 20, 89–108.PubMedCrossRefGoogle Scholar
  39. Katan, M. B. (2004). Commentary: Mendelian randomization, 18 years on. International Journal of Epidemiology, 33(1), 10–11.PubMedCrossRefGoogle Scholar
  40. Kaufman, J. S., & Cooper, R. S. (2001). Commentary: Considerations for use of racial/ethnic classification in etiologic research. American Journal of Epidemiology, 154(4), 291–298.PubMedCrossRefGoogle Scholar
  41. Kaufman, J., Maclehose, R., et al. (2004). A further critique of the analytic strategy of adjusting for covariates to identify biologic mediation. Epidemiologic Perspectives & Innovations, 1(1), 4.CrossRefGoogle Scholar
  42. Kennedy, P. (1998). A guide to econometrics. Cambridge: MIT.Google Scholar
  43. Lawlor, D. A., Harbord, R. M., et al. (2008). Mendelian randomization: Using genes as instruments for making causal inferences in epidemiology. Statistics in Medicine, 27(8), 1133–1163.PubMedCrossRefGoogle Scholar
  44. Little, J., & Khoury, M. J. (2003). Mendelian randomisation: a new spin or real progress. The Lancet, 362(9388), 930–931.CrossRefGoogle Scholar
  45. Little, R. J., & Rubin, D. B. (2002). Statistical analysis with missing data. Hoboken: Wiley.Google Scholar
  46. MacKinnon, D. P., Lockwood, C. M., et al. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7(1), 83–104.PubMedCrossRefGoogle Scholar
  47. Manly, J. J., Jacobs, D. M., et al. (1999). Effect of literacy on neuropsychological test performance in nondemented, education-matched elders. Journal of the International Neuropsychological Society, 5(3), 191–202.PubMedCrossRefGoogle Scholar
  48. Manly, J. J., Jacobs, D. M., et al. (2002). Reading level attenuates differences in neuropsychological test performance between African American and White elders (vol. 8, pp. 341–348). Cambridge: Cambridge University Press.Google Scholar
  49. Marang-van de Mheen, P. J., Shipley, M. J., et al. (2001). Decline of the relative risk of death associated with low employment grade at older age: the impact of age related differences in smoking, blood pressure and plasma cholesterol. Journal of Epidemiology and Community Health, 55(1), 24–28.PubMedCrossRefGoogle Scholar
  50. Mark, S. D., & Robins, J. M. (1993). A method for the analysis of randomized trials with compliance information: an application to the Multiple Risk Factor Intervention Trial. Controlled Clinical Trials, 14(2), 79–97.PubMedCrossRefGoogle Scholar
  51. Markides, K. S., & Machalek, R. (1984). Selective survival, aging and society. Archives of Gerontology and Geriatrics, 3(3), 207–222.PubMedCrossRefGoogle Scholar
  52. McArdle, J. J., & Woodcock, R. W. (1997). Expanding test–retest designs to include developmental time-lag components. Psychological Methods, 2(4), 403–435.CrossRefGoogle Scholar
  53. McCaffrey, R. J., & Westervelt, H. J. (1995). Issues associated with repeated neuropsychological assessments. Neuropsychology Review, 5(3), 203–221.PubMedCrossRefGoogle Scholar
  54. Mohtashemi, M., & Levins, R. (2002). Qualitative analysis of the all-cause Black–White mortality crossover. Bulletin of Mathematical Biology, 64(1), 147–173.PubMedCrossRefGoogle Scholar
  55. Nybo, H., Petersen, H. C., et al. (2003). Predictors of mortality in 2,249 nonagenarians—The Danish 1905-cohort survey. Journal of the American Geriatrics Society, 51(10), 1365–1373.PubMedCrossRefGoogle Scholar
  56. Oakes, J. M. (2004). The (mis)estimation of neighborhood effects: Causal inference for a practicable social epidemiology. Social Science & Medicine, 58(10), 1929–1952.CrossRefGoogle Scholar
  57. Paneth, N., & Susser, M. (1995). Early origin of coronary heart-disease (the Barker hypothesis). British Medical Journal, 310(6977), 411–412.PubMedGoogle Scholar
  58. Pearl, J. (2000). Causality. Cambridge: Cambridge University Press.Google Scholar
  59. Peto, R. (1981). The horse-racing effect [letter]. Lancet, 2(8244), 467–468.PubMedCrossRefGoogle Scholar
  60. Powell, J. L. (1984). Least absolute deviations estimation for the censored regression model. Journal of Econometrics, 25(3), 303–325.CrossRefGoogle Scholar
  61. Robins, J. M., & Greenland, S. (1992). Identifiability and exchangeability for direct and indirect effects. Epidemiology, 3(2), 143–155.PubMedCrossRefGoogle Scholar
  62. Robins, J. M., & Greenland, S. (1996). Identification of causal effects using instrumental variables—Comment. Journal of the American Statistical Association, 91(434), 456–458.CrossRefGoogle Scholar
  63. Rosenfeld, C. S., & Roberts, R. M. (2004). Maternal diet and other factors affecting offspring sex ratio: A review. Biology of Reproduction, 71(4), 1063–1070.PubMedCrossRefGoogle Scholar
  64. Rousseeuw, P. J., & Leroy, A. M. (1987). Robust regression and outlier detection. New York: Wiley.Google Scholar
  65. Rubin, D. B. (1997). Estimating causal effects from large data sets using propensity scores. Annals of Internal Medicine, 127(8), 757–763.PubMedGoogle Scholar
  66. Rust, K., & Rao, J. (1996). Variance estimation for complex surveys using replication techniques. Statistical Methods in Medical Research, 5(3), 283–310.PubMedCrossRefGoogle Scholar
  67. Salthouse, T. A., Schroeder, D. H., et al. (2004). Estimating retest effects in longitudinal assessments of cognitive functioning in adults between 18 and 60 years of age. Developmental Psychology, 40(5), 813–822.PubMedCrossRefGoogle Scholar
  68. Schaie, K. W. (1992). The impact of methodological changes in gerontology. International Journal of Aging & Human Development, 35(1), 19–29.CrossRefGoogle Scholar
  69. Schaie, K. W., & Hofer, S. M. (2001). Longitudinal studies in aging research. In J. E. Birren, & K. W. Schaie (Eds.), Handbook of the psychology of aging (pp. 53–77). San Diego: Academic.Google Scholar
  70. Smith, G. D. (2004). Genetic epidemiology: An ‘enlightened narrative’. International Journal of Epidemiology, 33(5), 923–924.CrossRefGoogle Scholar
  71. Smith, G. D., & Ebrahim, S. (2004). Mendelian randomization: prospects, potentials, and limitations. International Journal of Epidemiology, 33(1), 30–42.PubMedCrossRefGoogle Scholar
  72. Spirtes, P., Glymour, C., et al. (2001). Causation, prediction, and search. Cambridge: MIT.Google Scholar
  73. Tate, R. B., Manfreda, J., et al. (1998). The effect of age on risk factors for ischemic heart disease: The Manitoba Follow Up Study, 1948–1993. Annals of Epidemiology, 8(7), 415–421.PubMedCrossRefGoogle Scholar
  74. Thornton, R. (2004). The Navajo-US population mortality crossover since the mid-20th century. Population Research and Policy Review, 23(3), 291–308.CrossRefGoogle Scholar
  75. Thorvaldsson, V., Hofer, S. A., et al. (2006). Effects of repeated testing in a longitudinal age-homogeneous study of cognitive aging. Journals of Gerontology Series B-Psychological Sciences and Social Sciences, 61(6), P348–P354.Google Scholar
  76. Tombaugh, T. N. (2004). Trail making test A and B: Normative data stratified by age and education. Archives of Clinical Neuropsychology, 19(2), 203–214.PubMedCrossRefGoogle Scholar
  77. Tu, Y. K., West, R., et al. (2005). Why evidence for the fetal origins of adult disease might be a statistical artifact: The “reversal paradox” for the relation between birth weight and blood pressure in later life. American Journal of Epidemiology, 161(1), 27–32.PubMedCrossRefGoogle Scholar
  78. Weinberg, C. R. (2005). Invited commentary: Barker meets Simpson. American Journal of Epidemiology, 161(1), 33–35.PubMedCrossRefGoogle Scholar
  79. Wesnes, K., & Pincock, C. (2002). Practice effects on cognitive tasks: a major problem? Lancet Neurology, 1(8), 473.PubMedCrossRefGoogle Scholar
  80. Williams, J. E., Massing, M., et al. (1999). Racial disparities in CHD mortality from 1968–1992 in the state economic areas surrounding the ARIC study communities. Annals of Epidemiology, 9(8), 472–480.PubMedCrossRefGoogle Scholar
  81. Winship, C., & Morgan, S. L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, 659–706.CrossRefGoogle Scholar
  82. Wolfe, C. D. A., Smeeton, N. C., et al. (2005). Survival differences after stroke in a multiethnic population: follow-up study with the south London stroke register. British Medical Journal, 331(7514), 431–433.PubMedCrossRefGoogle Scholar
  83. Yanez, N. D., Kronmal, R. A., et al. (1998). The effects of measurement error in response variables and tests of association of explanatory variables in change models. Statistics in Medicine, 17(22), 2597–2606.PubMedCrossRefGoogle Scholar
  84. Yanez, N. D., Kronmal, R. A., et al. (2002). A regression model for longitudinal change in the presence of measurement error. Annals of Epidemiology, 12(1), 34–38.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • M. Maria Glymour
    • 1
    • 2
    Email author
  • Jennifer Weuve
    • 3
    • 4
  • Jarvis T. Chen
    • 1
  1. 1.Department of Society, Human Development, and HealthHarvard School of Public HealthBostonUSA
  2. 2.Department of Epidemiology, Mailman School of Public HealthColumbia UniversityNew YorkUSA
  3. 3.Department of Internal Medicine, Rush Institute for Healthy AgingRush University Medical CenterChicagoUSA
  4. 4.Department of Environmental HealthHarvard School of Public HealthBostonUSA

Personalised recommendations