Prevention Science

, Volume 9, Issue 4, pp 288–298 | Cite as

Estimating Intervention Effects of Prevention Programs: Accounting for Noncompliance

  • Elizabeth A. StuartEmail author
  • Deborah F. Perry
  • Huynh-Nhu Le
  • Nicholas S. Ialongo


Individuals not fully complying with their assigned treatments is a common problem encountered in randomized evaluations of behavioral interventions. Treatment group members rarely attend all sessions or do all “required” activities; control group members sometimes find ways to participate in aspects of the intervention. As a result, there is often interest in estimating both the effect of being assigned to participate in the intervention, as well as the impact of actually participating and doing all of the required activities. Methods known broadly as “complier average causal effects” (CACE) or “instrumental variables” (IV) methods have been developed to estimate this latter effect, but they are more commonly applied in medical and treatment research. Since the use of these statistical techniques in prevention trials has been less widespread, many prevention scientists may not be familiar with the underlying assumptions and limitations of CACE and IV approaches. This paper provides an introduction to these methods, described in the context of randomized controlled trials of two preventive interventions: one for perinatal depression among at-risk women and the other for aggressive disruptive behavior in children. Through these case studies, the underlying assumptions and limitations of these methods are highlighted.


Complier average causal effect Dosage effects Instrumental variables Randomized controlled trials 



This research supported in part by grant R40 MC 02497 from the Maternal and Child Health Bureau (Title V, Social Security Act), Health Resources and Services Administration, Department of Health and Human Services (PI: Le) as well as by the Center for Prevention and Early Intervention, jointly funded by the National Institute of Mental Health and the National Institute on Drug Abuse (MH066247; PI: Ialongo).


  1. Angrist, J., & Imbens, G. W. (1995). Two-stage least squares estimation of average causal effects in models with variable treatment intensity. Journal of the American Statistical Association, 90, 431–442. doi: 10.2307/2291054.CrossRefGoogle Scholar
  2. Angrist, J., Imbens, G. W., & Rubin, D. B. (1996). Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91, 444–472. doi: 10.2307/2291629.CrossRefGoogle Scholar
  3. Baker, S. G., & Kramer, B. S. (2005). Simple maximum likelihood estimates of efficacy in randomized trials and before-and-after studies, with implications for meta-analysis. Statistical Methods in Medical Research, 14, 349–367. doi: 10.1191/0962280205sm404oa.PubMedCrossRefGoogle Scholar
  4. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Manual for the Beck Depression Inventory (2nd ed.). San Antonio, TX: The Psychological Corporation.Google Scholar
  5. Black, M. M., Bentley, M. E., Papas, M. A., Oberlander, S., Teti, L. O., McNary, S., et al. (2006). Delaying second births among adolescent mothers: A randomized, controlled trial of a home-based mentoring program. Pediatrics, 118, e1087–e1099. doi: 10.1542/peds.2005-2318.PubMedCrossRefGoogle Scholar
  6. Bloom, H. S. (1984). Accounting for no-shows in experimental evaluation designs. Evaluation Review, 8, 225–246. doi: 10.1177/0193841X8400800205.CrossRefGoogle Scholar
  7. Connell, A. M., Dishion, T. J., Yasui, M., & Kavanagh, K. (2007). An adaptive approach to family intervention: Linking engagement in family-centered intervention to reductions in adolescent problem behavior. Journal of Consulting and Clinical Psychology, 75, 568–579. doi: 10.1037/0022-006X.75.4.568.PubMedCrossRefGoogle Scholar
  8. Dunn, G., Maracy, M., Dowrick, C., Ayuso-Mateos, J. L., Dalgard, O. S., Page, H., et al. (2003). Estimating psychological treatment effects from a randomised controlled trial with both non-compliance and loss to follow-up. The British Journal of Psychiatry, 183, 323–331. doi: 10.1192/bjp.183.4.323.PubMedCrossRefGoogle Scholar
  9. Dunn, G., Maracy, M., & Tomenson, B. (2005). Estimating treatment effects from randomized controlled trials with noncompliance and loss to follow-up: The role of instrumental variables methods. Statistical Methods in Medical Research, 14, 369–395. doi: 10.1191/0962280205sm403oa.PubMedCrossRefGoogle Scholar
  10. Foster, E. M. (2000). Is more better than less? An analysis of children’s mental health services. Health Services Research, 35, 1135–1158.PubMedGoogle Scholar
  11. Foster, E. M. (2003). Propensity score matching: An illustrative analysis of dose response. Medical Care, 41, 1183–1192. doi: 10.1097/01.MLR.0000089629.62884.22.PubMedCrossRefGoogle Scholar
  12. Frangakis, C. E., & Rubin, D. B. (1999). Addressing complications of intention-to-treat analysis in the combined presence of all-or-none treatment noncompliance and subsequent missing outcomes. Biometrika, 86, 365–379. doi: 10.1093/biomet/86.2.365.CrossRefGoogle Scholar
  13. Frangakis, C. E., & Rubin, D. B. (2002). Principal stratification in causal inference. Biometrics, 58, 21–29. doi: 10.1111/j.0006-341X.2002.00021.x.PubMedCrossRefGoogle Scholar
  14. Hill, J. L., Brooks-Gunn, J., & Waldfogel, J. B. B. (2003). Sustained effects of high participation in an early intervention for low-birth-weight premature infants. Developmental Psychology, 39, 730–744. doi: 10.1037/0012-1649.39.4.730.PubMedCrossRefGoogle Scholar
  15. Hirano, K., Imbens, G. W., Rubin, D. B., & Zhou, X. (2000). Assessing the effect of influenza vaccine in an encouragement design with covariates. Biostatistics (Oxford, England), 1, 69–88. doi: 10.1093/biostatistics/1.1.69.Google Scholar
  16. Ialongo, N., Werthamer, L., Brown, C. H., Kellam, S., & Wai, S. B. (1999). The proximal impact of two first grade preventive interventions on the early risk behaviors for later substance abuse, depression and antisocial behavior. American Journal of Community Psychology, 27, 599–642. doi: 10.1023/A:1022137920532.PubMedCrossRefGoogle Scholar
  17. Imbens, G. W., & Rubin, D. B. (1997). Bayesian inference for causal effects in randomized experiments with noncompliance. The Annals of Statistics, 25, 305–327. doi: 10.1214/aos/1034276631.CrossRefGoogle Scholar
  18. Jin, H., & Rubin, D. B. (2008). Principal stratification for causal inference with extended partial compliance. Journal of the American Statistical Association, 103, 101–111. doi: 10.1198/016214507000000347.CrossRefGoogle Scholar
  19. Jo, B. (2002a). Estimation of intervention effects with noncompliance: Alternative model specifications. Journal of Educational and Behavioral Statistics, 27, 385–409. doi: 10.3102/10769986027004385.CrossRefGoogle Scholar
  20. Jo, B. (2002b). Statistical power in randomized intervention studies with noncompliance. Psychological Methods, 7, 178–193. doi: 10.1037/1082-989X.7.2.178.PubMedCrossRefGoogle Scholar
  21. Jo, B., Asparouhov, T., Muthen, B. O., Ialongo, N. S., & Brown, C. H. (2008). Cluster randomized trials with treatment noncompliance. Psychological Methods, 13, 1–18. doi: 10.1037/1082-989X.13.1.1.PubMedCrossRefGoogle Scholar
  22. Kam, C.-M., Greenberg, M. T., & Walls, C. T. (2003). Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prevention Science, 4, 55–63. doi: 10.1023/A:1021786811186.PubMedCrossRefGoogle Scholar
  23. Lin, J. Y., Ten Have, T. R., & Elliott, M. R. (2008). Longitudinal nested compliance class model in the presence of time-varying noncompliance. Journal of the American Statistical Association, 103, 462–473.CrossRefGoogle Scholar
  24. Little, R. J., & Rubin, D. B. (2000). Causal effects in clinical and epidemiological studies via potential outcomes: Concepts and analytical approaches. Annual Review of Public Health, 21, 121–145. doi: 10.1146/annurev.publhealth.21.1.121.PubMedCrossRefGoogle Scholar
  25. Little, R. J. A., & Yau, L. (1998). Statistical techniques for analyzing data from prevention trials: Treatment of no-shows using Rubin’s causal model. Psychological Methods, 3, 147–159. doi: 10.1037/1082-989X.3.2.147.CrossRefGoogle Scholar
  26. Muñoz, R. F., Le, H. N., Ghosh Ippen, C., Diaz, M. A., Urizar Jr, G. G., Soto, J., et al. (2007). Prevention of postpartum depression in low-income women: Development of the Mamás y Bebés/Mothers and Babies Course. Cognitive and Behavioral Practice, 14, 70–83. doi: 10.1016/j.cbpra.2006.04.021.CrossRefGoogle Scholar
  27. Muthén, L. K., & Muthén, B. O. (1998–2007). Mplus user’s guide (5th ed.). Los Angeles, CA: Author.Google Scholar
  28. O’Malley, A. J., & Normand, S. T. (2005). Likelihood methods for treatment noncompliance and subsequent nonresponse in randomized trials. Biometrics, 61, 325–334. doi: 10.1111/j.1541-0420.2005.040313.x.PubMedCrossRefGoogle Scholar
  29. Peng, Y., Little, R. J., & Raghunathan, T. E. (2004). An extended general location model for causal inferences from data subject to noncompliance and missing values. Biometrics, 60, 598–607. doi: 10.1111/j.0006-341X.2004.00208.x.PubMedCrossRefGoogle Scholar
  30. R Core Development Team. (2007). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. ISBN 3-900051-07-0.
  31. Radloff, L. S. (1977). The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, 385–401. doi: 10.1177/014662167700100306.CrossRefGoogle Scholar
  32. Robins, J. M., & Greenland, S. (1992). Identifiability and exchangeability of direct and indirect effects. Epidemiology (Cambridge, Massachusetts), 3, 143–155. doi: 10.1097/00001648-199203000-00013.Google Scholar
  33. Rosenbaum, P. R. (2002). Observational studies (2nd ed.). New York: Springer.Google Scholar
  34. Rubin, D. B. (1978). Bayesian inference for causal effects: The role of randomization. The Annals of Statistics, 6, 34–58. doi: 10.1214/aos/1176344064.CrossRefGoogle Scholar
  35. SPSS Inc. (2007). SPSS Base 16.0 for Windows. Chicago, IL: Author.Google Scholar
  36. Stata Corp. (2007). Stata statistical software: Release 10. College Station, TX: Author.Google Scholar
  37. Werthamer-Larsson, L., Kellam, S. G., & Wheeler, L. (1991). Effect of first-grade classroom environment on child shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585–602. doi: 10.1007/BF00937993.PubMedCrossRefGoogle Scholar

Copyright information

© Society for Prevention Research 2008

Authors and Affiliations

  • Elizabeth A. Stuart
    • 1
    • 2
    Email author
  • Deborah F. Perry
    • 3
  • Huynh-Nhu Le
    • 4
  • Nicholas S. Ialongo
    • 1
  1. 1.Department of Mental HealthJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  2. 2.Department of BiostatisticsJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  3. 3.Department of Population, Family, and Reproductive HealthJohns Hopkins Bloomberg School of Public HealthBaltimoreUSA
  4. 4.Department of PsychologyThe George Washington UniversityWashingtonUSA

Personalised recommendations