Does Meeting the HEDIS Substance Abuse Treatment Engagement Criterion Predict Patient Outcomes?

  • Alex HS HarrisEmail author
  • Keith Humphreys
  • Thomas Bowe
  • Quyen Tiet
  • John W. Finney
Regular Article


This study examines the patient-level associations between the Health Plan Employer Data and Information Set (HEDIS) substance use disorder (SUD) treatment engagement quality indicator and improvements in clinical outcomes. Administrative and survey data from 2,789 US Department of Veterans Affairs SUD patients were used to estimate the effects of meeting the HEDIS engagement criterion on improvements in Addiction Severity Index Alcohol, Drug, and Legal composite scores. Patients meeting the engagement indicator improved significantly more in all domains than patients who did not engage, and the relationship was stronger for alcohol and legal outcomes for patients seen in outpatient settings. The benefit accrued by those who engaged was statistically significant but clinically modest. These results add to the literature documenting the clinical benefits of treatment entry and engagement. Although these findings only indirectly support the use of the HEDIS engagement measure for its intended purpose—discriminating quality at the facility or system level—they confirm that the processes of care captured by the measure are associated with important patient outcomes.


health care delivery substance abuse services veterans performance measures standards of quality HEDIS 



This study was funded by the Center for Substance Abuse Treatment (Contract 270-02-7120), and the VA Office of Research and Development Health Services Research and Development Service (grants no. SUS 99-015, MRP-05-168-1, IIR-07-092-1). Keith Humphreys is a member of the Washington Circle and Alex Harris is a member of the Washington Circle Public Sector Workgroup. The opinions expressed are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or the Center for Substance Abuse Treatment.


  1. 1.
    AcademyHealth. Glossary of terms commonly used in health care. Accessed April 22, 2008.
  2. 2.
    Garnick D, Horgan C, Chalk M. Performance Measures for Alcohol and Other Drug Services. National Institute of Alcohol Abuse and Alcoholism.–26.htm. Accessed September 12, 2006.
  3. 3.
    Donabedian A. The Definition of Quality and Approaches to Assessment. Ann Arbor: Health Administration Press; 1980.Google Scholar
  4. 4.
    Horn SD. Performance measures and clinical outcomes. JAMA. 2006;296(22):2731–2732. doi: 10.1001/jama.296.22.2731.CrossRefPubMedGoogle Scholar
  5. 5.
    Hofer TP, Hayward RA, Greenfield S, et al. The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease. JAMA. 1999;281(22):2098–2105. doi: 10.1001/jama.281.22.2098.CrossRefPubMedGoogle Scholar
  6. 6.
    Department of Veterans Affairs Office of Quality and Performance VA/DOD. Clinical Practice Guideline for the Management of Substance Use Disorders: Guideline Summary. Accessed July 26, 2005.
  7. 7.
    National Committee for Quality Assurance. HEDIS 2006 Volume 2: Technical Specifications. Washington, DC: National Committee for Quality Assurance; 2006.Google Scholar
  8. 8.
    Harris A, Humpheys K, Bowe T, et al. Measuring the quality of substance use disorder treatment: Assessing the VA continuity of care quality measure. Journal of Substance Abuse Treatment. in press.Google Scholar
  9. 9.
    Harris AH, Humphreys K, Finney JW. Veterans Affairs facility performance on Washington Circle indicators and case mix-adjusted effectiveness. Journal of Substance Abuse Treatment. 2007;33(4):333–339. doi: 10.1016/j.jsat.2006.12.015.CrossRefPubMedGoogle Scholar
  10. 10.
    Greenberg A, Rosenheck R, Seibyl C. Continuity of care and clinical effectiveness: Outcomes following residential treatment for severe substance abuse. Medical Care. 2002;40(3):246–259. doi: 10.1097/00005650-200203000-00008.CrossRefPubMedGoogle Scholar
  11. 11.
    Greenberg GA, Rosenheck RA. Continuity of care and clinical outcomes in a national health system. Psychiatric Services. 2005;56(4):427–433. doi: 10.1176/ Scholar
  12. 12.
    Rost K, Dickinson LM, Fortney J, et al. Clinical improvement associated with conformance to HEDIS-based depression care. Mental Health Services Research. 2005;7(2):103–112. doi: 10.1007/s11020-005-3781-1.CrossRefPubMedGoogle Scholar
  13. 13.
    McCorry F, Garnick DW, Bartlett J, et al. Developing performance measures for alcohol and other drug services in managed care plans. Joint Commission on Quality Improvement. 2000;26:633–643.Google Scholar
  14. 14.
    Garnick DW, Horgan C, Lee M, et al. Are Washington Circle performance measures associated with decreased criminal activity following treatment. Journal of Substance Abuse Treatment. 2007;33(4):341–352. doi: 10.1016/j.jsat.2007.03.002.CrossRefPubMedGoogle Scholar
  15. 15.
    Tiet QQ, Byrnes HF, Barnett P, et al. A practical system for monitoring the outcomes of substance use disorder patients. Journal of Substance Abuse Treatment. 2006;30(4):337–347. doi: 10.1016/j.jsat.2006.03.002.CrossRefPubMedGoogle Scholar
  16. 16.
    McLellan A, Kushner H, Metzger D, et al. The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment. 1992;9:199–213. doi: 10.1016/0740-5472(92)90062-S.CrossRefPubMedGoogle Scholar
  17. 17.
    Rosen CS, Henson BR, Finney JW, et al. Consistency of self-administered and interview-based Addiction Severity Index composite scores. Addiction. 2000;95(3):419–425. doi: 10.1046/j.1360-0443.2000.95341912.x.CrossRefPubMedGoogle Scholar
  18. 18.
    Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods. 1997;2(1):64–78. doi: 10.1037/1082-989X.2.1.64.CrossRefGoogle Scholar
  19. 19.
    Little RJA. Pattern-mixture models for multivariate incomplete data. Journal of the American Statistical Association. 1993;88:125–133. doi: 10.2307/2290705.CrossRefGoogle Scholar
  20. 20.
    Rubin DB. Inference and missing data. Biometrika. 1976;63:581–592. doi: 10.1093/biomet/63.3.581.CrossRefGoogle Scholar
  21. 21.
    Schafer JL, Graham JW. Missing data: our view of the state of the art. Psychological Methods. 2002;7(2):147–177. doi: 10.1037/1082-989X.7.2.147.CrossRefPubMedGoogle Scholar
  22. 22.
    Schafer JL. Analysis of Incomplete Multivariate Data. New York: Chapman & Hall; 1997.Google Scholar
  23. 23.
    Daniels M, Hogan J. Missing Data in Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. New York: Chapman & Hall; 2008.Google Scholar
  24. 24.
    R Foundation for Statistical Computing. R: A Language and Environment for Statistical Computing [computer program]. Vienna: R Foundation for Statistical Computing; 2005.Google Scholar
  25. 25.
    van Buuren S, Oudshoorn CGM. Multivariate Imputation by Chained Equations. R package version 1.14 [computer program]. 2005.Google Scholar
  26. 26.
    Glynn RJ, Laird NM, Rubin DB. Selection modeling versus mixture modeling with nonignorable nonresponse. In: Wainer H, ed. Drawing Inferences from Self-selected Samples. Mahwah: Lawrence Erlbaum Associates Publishers; 1986:115–142.Google Scholar
  27. 27.
    McCarty D. Performance measurement for systems treating alcohol and drug use disorders. Journal of Substance Abuse Treatment. 2007;33(4):353–354. doi: 10.1016/j.jsat.2007.04.002.CrossRefPubMedGoogle Scholar
  28. 28.
    Robinson W. Ecological correlations and the behavior of individuals. American Sociological Review. 1950;15:351–357. doi: 10.2307/2087176.CrossRefGoogle Scholar
  29. 29.
    Greenland S, Morgenstern H. Ecological bias, confounding, and effect modification. International Journal of Epidemiology. 1989;18(1):269–274. doi: 10.1093/ije/18.1.269.CrossRefPubMedGoogle Scholar
  30. 30.
    Werner RM, Bradlow ET. Relationship between Medicare’s hospital compare performance measures and mortality rates. JAMA. 2006;296(22):2694–2702. doi: 10.1001/jama.296.22.2694.CrossRefPubMedGoogle Scholar
  31. 31.
    McLellan A, Chalk M, Bartlett J. Outcomes, performance, and quality—what’s the difference. Journal of Substance Abuse Treatment. 2007;32:331–340. doi: 10.1016/j.jsat.2006.09.004.CrossRefPubMedGoogle Scholar
  32. 32.
    Witbrodt J, Bond J, Kaskutas LA, et al. Day hospital and residential addiction treatment: randomized and nonrandomized managed care clients. Journal of Consulting and Clinical Psychology. 2007;75(6):947–959. doi: 10.1037/0022-006X.75.6.947.CrossRefPubMedGoogle Scholar

Copyright information

© US Department of Veteran Affairs 2008

Authors and Affiliations

  • Alex HS Harris
    • 1
    Email author
  • Keith Humphreys
    • 1
  • Thomas Bowe
    • 1
  • Quyen Tiet
    • 1
  • John W. Finney
    • 1
  1. 1.Center for Health Care Evaluation, Department of Veterans AffairsPalo Alto Health Care System and Stanford University School of MedicineMenlo ParkUSA

Personalised recommendations