Health Services and Outcomes Research Methodology

, Volume 14, Issue 4, pp 232–254 | Cite as

Addressing confounding when estimating the effects of latent classes on a distal outcome

  • Megan S. SchulerEmail author
  • Jeannie-Marie S. Leoutsakos
  • Elizabeth A. Stuart


Confounding is widely recognized in settings where all variables are fully observed, yet recognition of and statistical methods to address confounding in the context of latent class regression are slowly emerging. In this study we focus on confounding when regressing a distal outcome on latent class; extending standard confounding methods is not straightforward when the treatment of interest is a latent variable. We describe a recent 1-step method, as well as two 3-step methods (modal and pseudoclass assignment) that incorporate propensity score weighting. Using simulated data, we compare the performance of these three adjusted methods to an unadjusted 1-step and unadjusted 3-step method. We also present an applied example regarding adolescent substance use treatment that examines the effect of treatment service class on subsequent substance use problems. Our simulations indicated that the adjusted 1-step method and both adjusted 3-step methods significantly reduced bias arising from confounding relative to the unadjusted 1-step and 3-step approaches. However, the adjusted 1-step method performed better than the adjusted 3-step methods with regard to bias and 95 % CI coverage, particularly when class separation was poor. Our applied example also highlighted the importance of addressing confounding—both unadjusted methods indicated significant differences across treatment classes with respect to the outcome, yet these class differences were not significant when using any of the three adjusted methods. Potential confounding should be carefully considered when conducting latent class regression with a distal outcome; failure to do so may results in significantly biased effect estimates or incorrect inferences.


Latent class Propensity scores Confounding Latent treatment 



The authors particularly thank Beth Ann Griffin for providing access to the data for the applied example (funded by award 1R01DA015697 from the National Institute of Drug Abuse (PI: McCaffrey) and Substance Abuse and Mental Health Services Administration (SAMHSA) contract #270-07-0191). Development of this article was funded by the Sommer Scholar program at Johns Hopkins, as well as awards P50DA010075 and T32DA017629 from the National Institute on Drug Abuse. Dr. Stuart’s time was partially supported by award 1R01MH099010 from the National Institute of Mental Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of NIDA, NIMH, SAMHSA, or NIH. The authors also thank the following Grantees and their participants for agreeing to share their data to support this secondary analysis: Assertive Adolescent Family Treatment (Study: AAFT; CSAT/SAMHSA contract #270-2003-00006 and #270-2007-00004C and Grantees: TI-17589, TI-17604, TI-17638, TI-17673, TI-17719, TI-17724, TI-17728, TI-17744, TI-17765, TI-17775, TI-17779, TI-17830, TI-17761, TI-17763, TI-17769, TI-17786, TI-17788, TI-17812, TI-17817, TI-17825, TI-17864), Adolescent Residential Treatment (Study: ART; CSAT/SAMHSA contracts #277-00-6500, #270-2003-00006 and Grantees: TI-14271, TI-14272, TI-14315, TI-14090, TI-14188, TI-14189, TI-14196, TI-14252, TI-14261, TI-14267, TI-14283, TI-14311, TI-14376), Adolescent Treatment Model (Study: ATM; CSAT/SAMHSA contracts #270-98-7047, #270-97-7011, #277-00-6500, #270-2003-00006 and Grantees: TI-11424, TI-11432, TI-11892, TI-11894), Cannabis Youth Treatment (Study: CYT; CSAT/SAMHSA contracts #270-97-7011, #270-00-6500, #270-2003-00006 and Grantees: TI-11317, TI-11321, TI-11323, TI-11324), Drug Court (Study: DC; CSAT/SAMHSA contract #270-2003-00006 and #270-2007-00004C and Grantees: TI-17433, TI-17475, TI-17484, TI-17517, TI-17434, TI-17446, TI-17486, TI-17523, TI-17535), Effective Adolescent Treatment (Study: EAT; CSAT/SAMHSA contract #270-2003-00006 and Grantees: TI-15413, TI-15415, TI-15421, TI-15433, TI-15438, TI-15446, TI-15447, TI-15458, TI-15461, TI-15466, TI-15467, TI-15469, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483, TI-15485, TI-15486, TI-15489, TI-15511, TI-15514, TI-15524, TI-15527, TI-15545, TI-15562, TI-15577, TI-15584, TI-15586, TI-15670, TI-15671, TI-15672, TI-15674, TI-15677, TI-15678, TI-15682, TI-15686), Strengthening Communities for Youth (study: SCY; CSAT/SAMHSA contracts #277-00-6500, #270-2003-00006 and Grantees: TI-13305, TI-13313, TI-13322, TI-13323, TI-13344, TI-13345, TI-13354, TI-13356), Targeted Capacity Expansion (Study: TCE and Grantees: TI-13190, TI-13601, TI-16386, TI-16400, TI-18406, TI-18723), and Young Offenders Reentry Program (Study: YORP; CSAT/SAMHSA contract #270-2003-00006 and #270-2007-00004C and Grantees: TI-16904, TI-16928, TI-16939, TI-16961, TI-16984, TI-16992, TI-17046, TI-17070, TI-17071, TI-19313).


  1. Asparouhov, T., Muthén, B. O.: Auxiliary variables in mixture modeling: A 3-step approach using Mplus. (2013). Accessed 01 Jan 2014
  2. Bakk, Z., Tekle, F., Vermunt, J.K.: Estimating the association between latent class membership and external variables using bias adjusted three-step approaches. Sociol. Methodol. 43, 272–311 (2013)CrossRefGoogle Scholar
  3. Bandeen-Roche, K., Miglioretti, D.L., Zeger, S.L., Rathouz, P.J.: Latent variable regression for multiple discrete outcomes. J. Am. Stat. Assoc. 92, 1375–1386 (1997)CrossRefGoogle Scholar
  4. Bolck, A., Croon, M.A., Hagenaars, J.A.: Estimating latent structure models with categorical variables: one-step versus three-step estimators. Polit Anal 12, 3–27 (2004)CrossRefGoogle Scholar
  5. Butera, N.M., Lanza, S.T., Coffman, D.L.: A framework for estimating causal effects in latent class analysis: Is there a causal link between early sex and subsequent profiles of delinquency? Prev. Sci. (2013). doi: 10.1007/s11121-013-0417-3
  6. Clogg, C.C.: Latent class models: recent developments and prospects for the future. In: Arminger, G., Clogg, C.C., Sobel, M.E. (eds.) Handbook of Statistical Modeling for the Social and Behavioral Sciences, pp. 311–352. Plenum, New York (1995)CrossRefGoogle Scholar
  7. Cole, S.R., Hernán, M.A.: Constructing inverse probability weights for marginal structural models. Am. J. Epidemiol. 168, 656–664 (2008)PubMedCentralPubMedCrossRefGoogle Scholar
  8. Collins, L.M., Lanza, S.T.: Latent Class and Latent Transition Analysis: With Applications in the Social, Behavioral, and Health Sciences. Wiley, New York (2010)Google Scholar
  9. Dennis, M.L., Dawud-Noursi, S., Muck, R.D., McDermeit, M.: The need for developing and evaluating adolescent treatment models. In: Stevens, S.J., Morral, A.R. (eds.) Adolescent Substance Abuse Treatment in the United States: Exemplary Models from a National Evaluation Study, pp. 3–34. The Haworth Press, Binghamton (2003)Google Scholar
  10. Dennis, M.L., Ives, M., Funk, R., Modisette, K., Bledsaw, R., Ihnes, P.: GAIN-I Encyclopedia of Supplemental Documentation on Scales and Other Calculated Variables. Chestnut Health Systems, Bloomington (2010)Google Scholar
  11. Dennis, M.L., Ives, M.L., White, M.K., Muck, R.D.: The Strengthening Communities for Youth (SCY) initiative: a cluster analysis of the services received, their correlates and how they are associated with outcomes. J. Psychoact Drugs 40, 3–16 (2008)CrossRefGoogle Scholar
  12. Dennis, M., Godley, S.H., Diamond, G., Tims, F.M., Babor, T., Donaldson, J., Liddle, H., Titus, J.C., Kaminer, Y., Webb, C., Hamilton, N., Funk, R.: The Cannabis Youth Treatment (CYT) study: main findings from two randomized trials. J. Subst. Abuse Treat. 27, 197–213 (2004)PubMedCrossRefGoogle Scholar
  13. Dennis, M.L., Titus, J.C., White, M., Unsicker, J., Hodgkins, D.: Global Appraisal of Individual Needs (GAIN): Administration Guide for the GAIN and Related Measures. Chestnut Health Systems. (2003). Accessed 01 Jan 2014
  14. Diamond, G., Godley, S.H., Liddle, H.A., Sampl, S., Webb, C., Tims, F.M., Meyers, R.: Five outpatient treatment models for adolescent marijuana use: a description of the Cannabis Youth Treatment Interventions. Addiction 97, 70–83 (2002)PubMedCrossRefGoogle Scholar
  15. Feingold, A., Tiberio, S.S., Capaldi, D.M.: New approaches for examining associations with latent categorical variables: Applications to substance abuse and aggression. Psychol. Addict. Behav. (2013). doi:  10.1037/a0031487
  16. Godley, M.D., Godley, S.H., Dennis, M.L., Funk, R.R., Passetti, L.L.: The effect of assertive continuing care on continuing care linkage, adherence and abstinence following residential treatment for adolescents with substance use disorders. Addiction 102, 81–93 (2007)PubMedCrossRefGoogle Scholar
  17. Goodman, L.A.: On the assignment of individuals to latent classes. Sociol. Methodol. 37, 1–22 (2007)CrossRefGoogle Scholar
  18. Graham, J.W., Olchowski, A.E., Gilreath, T.D.: How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prev. Sci. 8, 206–213 (2007)PubMedCrossRefGoogle Scholar
  19. Hagenaars, J.A., McCutcheon, A.L.: Applied Latent Class Analysis. Cambridge University Press, Cambridge (2002)CrossRefGoogle Scholar
  20. Ho, D.E., Imai, K., King, G., Stuart, E.A.: Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit Anal 15, 199–236 (2007)CrossRefGoogle Scholar
  21. Hochberg, Y.: A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 800–802 (1988)CrossRefGoogle Scholar
  22. Imbens, G.W.: The role of the propensity score in estimating dose-response functions. Biometrika 87(3), 706–710 (2000)CrossRefGoogle Scholar
  23. Kang, J., Schafer, J.L.: Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Stat. Sci. 22, 523–539 (2007)CrossRefGoogle Scholar
  24. Kang, J., Schafer, J.L.: Estimating average treatment effects when the treatment is a latent class. Technical Report 10-05. (2010). Accessed 01 Jan 2014
  25. Lanza, S.T., Tan, X., Bray, B.C.: Latent class analysis with distal outcomes: a flexible model-based approach. Struct. Eq. Modeling. 20, 1–26 (2013a)CrossRefGoogle Scholar
  26. Lanza, S.T., Coffman, D.L., Xu, S.: Causal inference in latent class analysis. Struct. Eq. Modeling. 20, 361–383 (2013b)CrossRefGoogle Scholar
  27. Lazarsfeld, P.F., Henry, N.W.: Latent Structure Analysis. Houghton Mifflin, Boston (1968)Google Scholar
  28. Lee, B.K., Lessler, J., Stuart, E.A.: Improving propensity score weighting using machine learning. Stat. Med. 29(3), 337–346 (2009)Google Scholar
  29. Lumley, T.: Analysis of complex survey samples. J. Stat. Softw. 9(1), 1–19 (2004)Google Scholar
  30. Lumley, T.: Package ‘survey.’ (2013). Accessed 01 Jan 2014
  31. Lunceford, J.K., Davidian, M.: Stratification and weighting via the propensity score in estimation of causal treatment effects: a comparative study. Stat. Med. 23(19), 2937–2960 (2004)PubMedCrossRefGoogle Scholar
  32. McCaffrey, D.F., Griffin, B.A., Almirall, D., Slaughter, M.E., Ramchand, R., Burgette, L.F.: A tutorial on propensity score estimation for multiple treatments using generalized boosted models. Stat. Med. 32(19), 3388–3414 (2013)PubMedCentralPubMedCrossRefGoogle Scholar
  33. McCaffrey, D.F., Ridgeway, G., Morral, A.R.: Propensity score estimation with boosted regression for evaluating causal effects in observational studies. Psychol. Methods 9, 403–425 (2004)PubMedCrossRefGoogle Scholar
  34. Melchior, L.A., Griffith, A.A., Huba, G.J.: Evaluation of the EFFECTIVE ADOLESCENT TREATMENT (EAT) program. The Measurement Group, LLC. (2007). Accessed 01 Jan 2014
  35. Nagin, D.S.: Group-Based Modeling of Development. Harvard University Press, Cambridge (2005)Google Scholar
  36. Petersen, J., Bandeen-Roche, K., Budtz-Jørgensen, E., Groes Larsen, K.: Predicting latent class scores for subsequent analysis. Psychometrika 77, 244–262 (2012)Google Scholar
  37. Petras, H., Masyn, K.: General growth mixture analysis with antecedents and consequences of change. In: Piquero, A.R., Weisburd, D. (eds.) Handbook of Quantitative Criminology, pp. 69–100. Springer, New York (2010)CrossRefGoogle Scholar
  38. R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. (2013). Accessed 01 Jan 2014
  39. Rosenbaum, P.R., Rubin, D.B.: The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)CrossRefGoogle Scholar
  40. Rosenbaum, P.R., Rubin, D.B.: Reducing bias in observational studies using subclassification on the propensity score. J. Amer. Statist. Assoc. 79, 516–524 (1984)CrossRefGoogle Scholar
  41. Rubin, D.B.: Multiple Imputation for Nonresponse in Surveys. J. Wiley & Sons, New York (1987)CrossRefGoogle Scholar
  42. Rubin, D.B.: Using propensity scores to help design observational studies: application to the tobacco litigation. Health Serv. Outcomes Res. Method. 2, 169–188 (2001)CrossRefGoogle Scholar
  43. Schafer, J.L., Kang, J.: LCCA package for R users’ guide (Version 1.1.0). The Methodology Center, Penn State. (2013a). Accessed 01 Jan 2014
  44. Schuler, M.S.: Estimating the relative treatment effects of natural clusters of adolescent substance abuse treatment services: Combining latent class analysis and propensity score methods. (Unpublished doctoral dissertation). Johns Hopkins Bloomberg School of Public Health, Baltimore (2013)Google Scholar
  45. Stuart, E.A.: Matching methods for causal inference: a review and a look forward. Stat. Sci. 25(1), 1–21 (2010)PubMedCentralPubMedCrossRefGoogle Scholar
  46. Substance Abuse and Mental Health Services Administration: Center for Substance Abuse Treatment: Grants to Improve the Quality and Availability of Residential Treatment and its Continuing Care Component for Adolescents. Request for Applications (RFA) no. TI-02-007. Part I-Programmatic guidance. (2002). Accessed 01 Jan 2014
  47. Substance Abuse and Mental Health Services Administration: Center for Substance Abuse Treatment: Adopt/expand effective adolescent alcohol and drug abuse treatment. Request for Applications (RFA) no. TI03-007. Part I-Programmatic guidance. (2003). Accessed 01 Jan 2014
  48. Substance Abuse and Mental Health Services Administration: Center for Substance Abuse Treatment: Substance Abuse Treatment and Reentry Services to Sentenced Juveniles and Young Adult Offenders Returning to the Community from the Correctional System. Request for Applications (RFA) no. TI04-001. Part I-Programmatic guidance. (2004). Accessed 01 Jan 2014
  49. Substance Abuse and Mental Health Services Administration: Center for Substance Abuse Treatment: Family and Juvenile Treatment Drug Courts. Request for Applications (RFA) no. TI05-005. Part I-Programmatic guidance. (2005). Accessed 01 Jan 2014
  50. Vermunt, J.K.: Latent class modeling with covariates: two improved three-step approaches. Polit Anal 18, 450–469 (2010)CrossRefGoogle Scholar
  51. Wang, C.P., Brown, C.H., Bandeen-Roche, K.: Residual diagnostics for growth mixture models: examining the impact of a preventive intervention on multiple trajectories of aggressive behavior. J. Am. Stat. Assoc. 100(471), 1054–1076 (2005)CrossRefGoogle Scholar
  52. Wang, C.Y., Huang, Y., Chao, E.C., Jeffcoat, M.K.: Expected estimating equations for missing data, measurement error, and misclassification, with application to longitudinal nonignorable missing data. Biometrics 64, 85–95 (2008)PubMedCrossRefGoogle Scholar
  53. Wilson, M.T., Atanda, R., Atkinson, D.D., Mulvey, K.: Outcomes from the targeted capacity expansion (TCE) substance abuse treatment program. Eval. Program Plann. 28, 341–348 (2005)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Megan S. Schuler
    • 1
    Email author
  • Jeannie-Marie S. Leoutsakos
    • 2
  • Elizabeth A. Stuart
    • 1
    • 3
  1. 1.Department of Mental HealthJohns Hopkins University Bloomberg School of Public HealthBaltimoreUSA
  2. 2.Department of Psychiatry and Behavioral SciencesJohns Hopkins University School of MedicineBaltimoreUSA
  3. 3.Department of BiostatisticsJohns Hopkins University School of MedicineBaltimoreUSA

Personalised recommendations