Prevention Science

, Volume 11, Issue 4, pp 384–396 | Cite as

Handling Missing Data in Randomized Experiments with Noncompliance

  • Booil JoEmail author
  • Elizabeth M. Ginexi
  • Nicholas S. Ialongo


Treatment noncompliance and missing outcomes at posttreatment assessments are common problems in field experiments in naturalistic settings. Although the two complications often occur simultaneously, statistical methods that address both complications have not been routinely considered in data analysis practice in the prevention research field. This paper shows that identification and estimation of causal treatment effects considering both noncompliance and missing outcomes can be relatively easily conducted under various missing data assumptions. We review a few assumptions on missing data in the presence of noncompliance, including the latent ignorability proposed by Frangakis and Rubin (Biometrika 86:365–379, 1999), and show how these assumptions can be used in the parametric complier average causal effect (CACE) estimation framework. As an easy way of sensitivity analysis, we propose the use of alternative missing data assumptions, which will provide a range of causal effect estimates. In this way, we are less likely to settle with a possibly biased causal effect estimate based on a single assumption. We demonstrate how alternative missing data assumptions affect identification of causal effects, focusing on the CACE. The data from the Johns Hopkins School Intervention Study (Ialongo et al., Am J Community Psychol 27:599–642, 1999) will be used as an example.


Causal inference Complier average causal effect Latent ignorability Missing at random Missing data Noncompliance 


  1. Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–455.CrossRefGoogle Scholar
  2. Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246.CrossRefGoogle Scholar
  3. Dunn, G., Maracy, M., Dowrick, C., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., et al. (2003). Estimating psychological treatment effects from a randomized controlled trial with both non-compliance and loss to follow-up. British Journal of Psychiatry, 183, 323–331.CrossRefPubMedGoogle Scholar
  4. Emsley, R., Dunn, G., & White, I. R. (2010). Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Statistical Methods in Medical Research. doi: 10.1177/0962280209105014.
  5. Frangakis, C. E. & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the presence of all-or-none treatment-noncompliance and subsequent missing outcomes. Biometrika, 86, 365–379.CrossRefGoogle Scholar
  6. Frangakis, C. E. & Rubin, D. B. (2002) Principal stratification in causal inference. Biometrics, 58, 21–29.CrossRefPubMedGoogle Scholar
  7. Frangakis, C. E., Rubin, D. B., & Zhou, X. H. (2002). Clustered encouragement design with individual noncompliance: Bayesian inference and application to advance directive forms. Biostatistics, 3, 147–164.CrossRefPubMedGoogle Scholar
  8. Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X. H. (2000). Assessing the effect of an influenza vaccine in an encouragement design. Biostatistics, 1, 69–88.CrossRefPubMedGoogle Scholar
  9. Holland, P. W. (1986). Statistics and causal inference. Journal of the American Statistical Association, 81, 945–960.CrossRefGoogle Scholar
  10. Ialongo, N. S., Werthamer, L., Kellam, S. G., Brown, C. H., Wang, S., & Lin, Y. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology, 27, 599–642.CrossRefPubMedGoogle Scholar
  11. Imbens, G. W. & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with non-compliance. Annals of Statistics, 25, 305–327.CrossRefGoogle Scholar
  12. Jo, B. (2002a). Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7, 178–193.CrossRefPubMedGoogle Scholar
  13. Jo, B. (2002b). Estimating intervention effects with noncompliance: Alternative model specifications. Journal of Educational and Behavioral Statistics, 27, 385–420.CrossRefGoogle Scholar
  14. Jo, B. (2008a). Bias mechanisms in intention-to-treat analysis with data subject to treatment noncompliance and missing outcomes. Journal of Educational and Behavioral Statistics, 33, 158–185.CrossRefGoogle Scholar
  15. Jo, B. (2008b). Causal inference in randomized experiments with mediational processes. Psychological Methods, 13, 314–336.CrossRefPubMedGoogle Scholar
  16. Jo, B., Asparouhov, T., Muthén, B. O., Ialongo, N. S., & Brown, C. H. (2008). Cluster randomized trials with treatment noncompliance. Psychological Methods, 13, 1–18.CrossRefPubMedGoogle Scholar
  17. Jo, B., & Vinokur, A. (2010). Sensitivity analysis and bounding of causal effects with alternative identifying assumptions. Journal of Educational and Behavioral Statistics, in press.Google Scholar
  18. Kellam, S. G., Branch, J. D., Agrawal, K. C., & Ensminger, M. E. (1975). Mental health and going to school: The Woodlawn program of assessment, early intervention, and evaluation. Chicago: University of Chicago Press.Google Scholar
  19. Little, R. J. A. & Rubin, D. B. (2002). Statistical analysis with missing data. New York: Wiley.Google Scholar
  20. Little, R. J. A., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin’s causal model. Psychological Methods, 3, 147–159.CrossRefGoogle Scholar
  21. Mattei, A., & Mealli, F. (2007). Application of the principal stratification approach to the Faenza randomized experiment on breast self-examination. Biometrics, 63, 437–446.CrossRefPubMedGoogle Scholar
  22. Mealli, F., Imbens, G. W., Ferro, S., & Biggeri A. (2004). Analyzing a randomized trial on breast self-examination with noncompliance and missing outcomes. Biostatistics, 5, 207–222.CrossRefPubMedGoogle Scholar
  23. Muthén, L. K., & Muthén, B. O. (1998–2009). Mplus user’s guide. Los Angeles: Muthén & Muthén.Google Scholar
  24. Neyman, J. (1923). On the application of probability theory to agricultural experiments. Section 9 translated in Statistical Science, 5, 465–480 (1990).Google Scholar
  25. O’Malley, A. J., & Normand, S. L. T. (2004). Likelihood methods for treatment noncompliance and subsequent nonresponse in randomized trials. Biometrics, 61, 325–334.CrossRefGoogle Scholar
  26. Peng, Y., Little, R. J., & Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics, 60, 598–607.CrossRefPubMedGoogle Scholar
  27. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. Annals of Statistics, 6, 34–58.CrossRefGoogle Scholar
  28. Rubin, D. B. (1980). Discussion of “randomization analysis of experimental data in the Fisher randomization test” by D. Basu. Journal of the American Statistical Association, 75, 591–593.CrossRefGoogle Scholar
  29. Rubin, D. B. (1990). Comment on “Neyman (1923) and causal inference in experiments and observational studies.” Statistical Science, 5, 472–480.Google Scholar
  30. Schafer, J. L. (1997). Analysis of incomplete multivariate data. London: CRC.CrossRefGoogle Scholar
  31. Sobel, M. E. (2006).What do randomized studies of housing mobility demonstrate: Causal inference in the face of interference. Journal of the American Statistical Association, 101, 1398–1407.CrossRefGoogle Scholar
  32. Stuart, E. A., Perry, D. F., Le, H-N, & Ialongo, N. S. (2008). Estimating intervention effects of prevention programs: Accounting for noncompliance. Prevention Science, 9, 288–298.CrossRefPubMedGoogle Scholar
  33. Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602.CrossRefPubMedGoogle Scholar

Copyright information

© Society for Prevention Research 2010

Authors and Affiliations

  • Booil Jo
    • 1
    Email author
  • Elizabeth M. Ginexi
    • 2
  • Nicholas S. Ialongo
    • 3
  1. 1.Department of Psychiatry & Behavioral SciencesStanford UniversityStanfordUSA
  2. 2.Center for Family ResearchGeorge Washington UniversityWashingtonUSA
  3. 3.Department of Mental HealthJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations