Skip to main content

Advertisement

SpringerLink
Log in

Clinically Significant Change to Establish Benchmarks in Residential Drug and Alcohol Treatment Services

  • Published:
International Journal of Mental Health and Addiction Aims and scope Submit manuscript

Abstract

There is increasing emphasis on the use routine outcome assessment measures to inform quality assurance initiatives. The calculation of reliable and clinically significant change indices is one strategy that organizations could use to develop both internal and externally focused benchmarking processes. The current study aimed to develop reliable and clinically significant change indices for a range of commonly used outcome measures in the substance abuse field. Participants were attending Kedesh Rehabilitation Services, an Australian residential substance abuse service that also accepts people with co-occurring mental health problems (n = 595). A range of mental health and substance use measures were administered to participants at intake to the service, and just prior to discharge at 7-weeks. Participants demonstrated statistically significant improvement across each of the measures. Rates of clinically significant improvement varied from 44 % to 85 % dependent on the criteria and outcome measure used. The calculation of clinically significant change provides a low cost, and relatively straightforward approach to evaluating and benchmarking program activities. It is likely that the results from the current study will be beneficial for other residential substance abuse services looking to benchmark their own program activities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Anand, G., & Kodali, R. (2008). Benchmarking the benchmarking models. Benchmarking: An international journal, 15(3), 257–291.

    Article  Google Scholar 

  • Audin, K., Margison, F. R., Clark, J. M., & Barkham, M. (2001). Value of HoNOS in assessing patient change in NHS psychotherapy and psychological treatment services. The British Journal of Psychiatry, 178, 561–566.

    Article  PubMed  CAS  Google Scholar 

  • Bayney, R. (2005). Benchmarking in mental health: an introduction for psychiatrists. Journal of Continuing Professional Development, 11, 305–314.

    Google Scholar 

  • Bowersox, N., Saunders, S., & Wojcik, J. (2009). An evaluation of the utility of statistical versus clinical significance in determining improvement in alcohol and other drug (AOD) treatment in correctional settings. Alcoholism Treatment Quarterly, 27, 113–129.

    Article  Google Scholar 

  • Bringhurst, D. L. (2004). The reliability and validity of the outcome rating scale: a replication study of a brief clinical measure. Journal of Brief Therapy, 1–8.

  • Brown, T. A., Chorpita, B. F., Korotitsch, W., & Barlow, D. H. (1997). Psychometric properties of the Depression Anxiety Stress Scales (DASS) in clinical samples. Behaviour Research and Therapy, 35(1), 79–89.

    Article  PubMed  CAS  Google Scholar 

  • Campbell, A., & Hemsley, S. (2011). Outcome Rating Scale and Session Rating Scale in psychological practice: Clinical utility of ultra-brief measures. Clinical Psychologist, 13(1), 1–9.

    Article  Google Scholar 

  • Christensen, L., & Mendoza, J. L. (1986). A method of assessing change in a single subject - an alteration of the Rc index. Behavior Therapy, 17(3), 305–308.

    Article  Google Scholar 

  • Crawford, J. H. (2003). The Depression Anxiety Stress Scales (DASS): normative data and latent structure in a large non-clinical sample. The British Journal of Clinical Psychology, 42, 111–131.

    Article  PubMed  Google Scholar 

  • Crawford, J. R., & Henry, J. D. (2003). The Depression Anxiety Stress Scales (DASS): normative data and latent structure in a large non-clinical sample. British Journal of Clinical Psychology, 42, 111–131.

    Article  PubMed  Google Scholar 

  • Goldberg, D. P. (1972). Manual of the general health questionnaire. London: Oxford University Press.

    Google Scholar 

  • Goldberg, D. P., & Williams, P. (1988). A user's guide to the General Health Questionnaire. Windsor: NFER-Nelson.

    Google Scholar 

  • Goldberg, D., Gater, R., Sartorius, N., Ustun, T., Piccinelli, M., Gureje, O., & Rutter, C. (1997). The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychological Medicine, 27, 191–197.

    Article  PubMed  CAS  Google Scholar 

  • Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W., Shimokawa, K., et al. (2005). A lab test and algorithms for identifying clients at risk for treatment failure. Journal of Clinical Psychology, 61(2), 155–163.

    Article  PubMed  Google Scholar 

  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.

    Article  PubMed  CAS  Google Scholar 

  • Kelly, P. J. (2010). Calculating clinically significant change: applications of the Clinical Global Impressions (CGI) scale to evaluate client outcomes in private practice. Clinical Psychologist, 14, 107–111.

    Article  Google Scholar 

  • Lambert, M., & Ogles, B. M. (2009). Using clinical significance in psychotherapy outcome research: the need for a common procedure and validity data. Psychotherapy Research, 19(4–5), 493–501.

    Article  PubMed  Google Scholar 

  • Lovibond, P. (1995). The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy, 33(3), 335–343.

    Article  PubMed  CAS  Google Scholar 

  • Lovibond, S. H., & Lovibond, P. F. (1995). Manual for the Depression Anxiety Stress Scales (DASS). Sydney: Psychology Foundation.

    Google Scholar 

  • McKay, J., van Horn, D., & Oslin, D. (2010). A randomized trial of extended telephone-based continuing care for alcohol dependence: within-treatment substance use outcomes. Journal of Consulting & Clinical Psychology.

  • McLellan, A. T., McKay, J. R., Forman, R., Cacciola, J., & Kemp, J. (2005). Reconsidering the evaluation of addiction treatment: from retrospective follow-up to concurrent recovery monitoring. Addiction, 100(4), 447–458.

    Article  PubMed  Google Scholar 

  • Mensinger, J. L., Lynch, K. G., TenHave, T. R., & McKay, J. R. (2007). Mediators of telephone-based continuing care for alcohol and cocaine dependence. Journal of Consulting and Clinical Psychology, 75(5), 775–784.

    Article  PubMed  Google Scholar 

  • Merrill, K. A., Tolbert, V. E., & Wade, W. A. (2003). Effectiveness of cognitive therapy for depression in a community mental health center: a benchmarking study. Journal of Consulting and Clinical Psychology, 71(2), 404–409.

    Article  PubMed  Google Scholar 

  • Miller, W. R., & Manuel, J. K. (2008). How large must a treatment effect be before it matters to practitioners? An estimation method and demonstration. Drug and Alcohol Review, 27(5), 524–528.

    Article  PubMed  Google Scholar 

  • Murugesan, G., Amey, C. G., Deane, F. P., Jeffrey, R., Kelly, B., & Stain, H. (2007). Inpatient psychosocial rehabilitation in rural NSW: assessment of clinically significant change for people with severe mental illness. The Australian and New Zealand Journal of Psychiatry, 41(4), 343–350.

    Article  PubMed  Google Scholar 

  • NSW Health. (2003). Your guide to MH-OAT: Clinicians' reference guide to NSW Mental Health Outcomes and Assessment Tools. Sydney, Australia.

  • Oei, T. P. S., & Boschen, M. J. (2009). Clinical effectiveness of a cognitive behavioral group treatment program for anxiety disorders: a benchmarking study. Journal of Anxiety Disorders, 23(7), 950–957.

    Article  PubMed  Google Scholar 

  • Orrell, M., Yard, P., Handysides, J., & Schapira, R. (1999). Validity and reliability of the Health of the Nation Outcome Scales in psychiatric patients in the community. The British Journal of Clinical Psychology, 174, 409–412.

    CAS  Google Scholar 

  • Scott, C. K., & Dennis, M. L. (2009). Results from two randomized clinical trials evaluating the impact of quarterly recovery management checkups with adult chronic substance users. Addiction, 104(6), 959–971.

    Article  PubMed  Google Scholar 

  • Sklar, S. M., & Turner, N. E. (1999). A brief measure for the assessment of coping self-efficacy among alcohol and other drug users. Addiction, 94(5), 723–729.

    Article  PubMed  CAS  Google Scholar 

  • Sklar, S., Annis, H., & Turner, N. (1997). Development and Validation of the Drug-Taking Confidence Questionnaire: a measure of coping self-efficacy. Addictive Behaviors, 22(5), 655–670.

    Article  PubMed  CAS  Google Scholar 

  • Slade, M. (2002). Routine outcome assessment in mental health services. Psychological Medicine, 32, 1339–1343.

    PubMed  Google Scholar 

  • Trauer, T. (1999). The subscale structure of the Health of the Nation Outcome Scales (HoNOS). Journal of Mental Health, 8(5), 499–509.

    Article  Google Scholar 

  • Westbrook, D., & Kirk, J. (2004). The clinical effectiveness of cognitive behaviour therapy: outcome for a large sample of adults treated in routine practice. Behaviour Research and Therapy, 43, 1243–1261.

    Article  PubMed  Google Scholar 

  • Willmott, S. A., Boardman, J. A. P., Henshaw, C. A., & Jones, P. W. (2004). Understanding General Health Questionnaire (GHQ-28) score and its threshold. Social Psychiatry and Psychiatric Epidemiology, 39, 613–617.

    Article  PubMed  Google Scholar 

  • Wing, J. K., Curtis, R. H., & Beevor, A. S. (2000). Health of the Nation Outcome Scales (HoNOS). Washington: American Psychiatric Association.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Illawarra Institute for Mental Health, School of Psychology, University of Wollongong, Wollongong, NSW, Australia

    Daniel D. Billingham, Peter J. Kelly, Frank P. Deane & Trevor P. Crowe

  2. Kedesh Rehabilitation Services, Unanderra, Australia

    Mark S. Buckingham & Fiona L. Craig

  3. School of Psychology, University of Wollongong, Wollongong, NSW, Australia, 2500

    Peter J. Kelly

Authors
  1. Daniel D. Billingham
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter J. Kelly
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Frank P. Deane
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Trevor P. Crowe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mark S. Buckingham
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fiona L. Craig
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peter J. Kelly.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Billingham, D.D., Kelly, P.J., Deane, F.P. et al. Clinically Significant Change to Establish Benchmarks in Residential Drug and Alcohol Treatment Services. Int J Ment Health Addiction 10, 890–901 (2012). https://doi.org/10.1007/s11469-012-9384-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11469-012-9384-3

Keywords

Search

Navigation