Are Children Improving? Results from Outcome Measurement in a Large Mental Health System

Original Paper
First Online:

Abstract

Standardized outcome assessment was implemented in a large county mental health system. This study examines changes in children’s emotional and behavioral problems after 6 months of treatment and identifies factors associated with treatment improvement. Results from 3,215 youth indicated that externalizing and internalizing problems significantly decreased from intake to 6 months and that ethnicity, gender, and age were associated with caregiver-reported changes in symptomatology. While there was improvement associated with entry into treatment, the magnitude of symptom reduction was small to medium and not associated with amount of care, providing only limited support for the impact of routine out-patient care.

Keywords

Children’s mental health services Psychotherapy outcomes Usual care 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work was supported by a contract from San Diego County Department of Health and Human Services. The authors thank Bill Ganger, M.S. for data management and Jennifer Rolls-Reutz, MPH for project management.

References

  1. Achenbach, T. (1991). Manual for the child behavior checklist/4–18 and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry.Google Scholar
  2. Achenbach, T. M., & Edelbrock, C. S. (1979). The child behavior profile: II. Boys aged 12–16 and girls aged 6–11 and 12–16. Journal of Consulting and Clinical Psychology, 47, 223–233.PubMedCrossRefGoogle Scholar
  3. Alegria, M., Atkins, M., Farmer, E., Slaton, E., & Stelk, W. (2010). One size does not fit all: Taking diversity, culture, and context seriously. Administration and Policy in Mental Health and Mental Health Services Research, 37, 48–60.PubMedCrossRefGoogle Scholar
  4. Andrade, A. R., Lambert, W., & Bickman, L. (2000). Dose effect in child psychotherapy: Outcomes associated with negligible treatment. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 161–168.PubMedCrossRefGoogle Scholar
  5. Angold, A., Costello, J., Burns, B. J., Erkanli, A., & Farmer, E. M. Z. (2000). Effectiveness of nonresidential specialty mental health services for children and adolescents in the “real world”. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 154–160.PubMedCrossRefGoogle Scholar
  6. Bickman, L., Andrade, A. R., & Lambert, W. (2004). Dose response in child and adolescent mental health services. Mental Health Services Research, 4, 57–70.CrossRefGoogle Scholar
  7. Blais, M. A., Frank, R. G., Nierenberg, A. A., & Rauch, S. L. (2009). Measuring outcomes makes good sense. Psychiatric Services, 60, 112.PubMedCrossRefGoogle Scholar
  8. Brookman-Frazee, L., Haine, R. A., & Garland, A. F. (2006). Measuring outcomes of real-world youth psychotherapy: Whom to ask and what to ask? Psychiatric Services, 57, 1373–1375.PubMedCrossRefGoogle Scholar
  9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  10. Daleiden, E. L., Chorpita, B. F., Donkervoet, C. M., Arensdorf, A. A., & Brogan, M. (2006). Getting better at getting them better: Outcomes and evidence-based practice within a system of care. Journal of the American Academy of Child and Adolescent Psychiatry, 45, 749–756.PubMedCrossRefGoogle Scholar
  11. Doucette, A. (2004). Child/adolescent measurement system (CAMS): User manual. Nashville, TN.Google Scholar
  12. Faraone, S. V., Biederman, J., & Mick, E. (2006). The age-dependent decline of attention deficit hyperactivity disorder: A meta-analysis of follow-up studies. Psychological Medicine, 36, 159–165.PubMedCrossRefGoogle Scholar
  13. Foster, M. E., Kelsch, C. C., Kamradt, B., Sosna, T., & Yang, Z. (2001). Expenditures and sustainability in systems of care. Journal of Emotional and Behavioral Disorders, 9, 53–62.CrossRefGoogle Scholar
  14. Garland, A. F., Brookman-Frazee, L., Hurlburt, M. S., Accurso, E. C., Zoffness, R., Haine, R. A., & Ganger, W. (2010). Mental health care for children with disruptive behavior problems: A view inside therapists’ offices. Psychiatric Services, 61, 788–795.PubMedCrossRefGoogle Scholar
  15. Garland, B., & Chorpita, B. F. (2010). Change what? Identifying quality improvement targets by investigating usual mental health care. Administration and Policy in Mental Health and Mental Health Services Research. Advance online publication. doi: 10.1007/s10488-010-0279-y.
  16. Garland, A. F., Lau, A. S., Yeh, M., McCabe, K. M., Hough, R. L., & Landsverk, J. A. (2005). Racial and ethnic differences in utilization of mental health services among high-risk youths. American Journal of Psychiatry, 162, 1336–1343.PubMedCrossRefGoogle Scholar
  17. Gold, J., Buonopane, R. J., Caggiano, R. A., Picciotto, M., Vogeli, C., Kanner, N. T., et al. (2009). Assessing outcomes in child psychiatry. American Journal of Managed Care, 15, 210–216.PubMedGoogle Scholar
  18. Green, S. B., Salkind, N. J., & Akey, T. M. (2000). Using SPSS for windows: Analyzing and understanding data. New Jersey: Prentice Hall.Google Scholar
  19. Handwerk, M. L., Larzelere, R. E., Soper, S. H., & Friman, P. C. (1999). Parent and child discrepancies in reporting severity of problem behaviors in three out-of-home settings. Psychological Assessment, 11, 14–23.CrossRefGoogle Scholar
  20. Hankin, B. L., Abramson, L. Y., Silva, P. A., McGee, R., & Moffitt, T. E. (1998). Development of depression from preadolescence to young adulthood: Emerging gender differences in a 10-year longitudinal study. Journal of Abnormal Psychology, 107, 128–140.PubMedCrossRefGoogle Scholar
  21. Hatfield, D., & Ogles, B. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35, 485–491.CrossRefGoogle Scholar
  22. Hill, J. C., & Schoener, E. P. (1996). Age-dependent decline of attention deficit hyperactivity disorder. American Journal of Psychiatry, 153, 1143–1146.PubMedGoogle Scholar
  23. Hoagwood, K., & Kolko, D. (2009). Introduction to the Special Section on Practice Contexts: A Glimpse into the Nether World of Public Mental Health Services for Children and Families. Administration and Policy in Mental Health and Mental Health Services Research, 36, 35–36.PubMedCrossRefGoogle Scholar
  24. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12–19.PubMedCrossRefGoogle Scholar
  25. Kataoka, S. H., Zhang, L., & Wells, K. B. (2002). Unmet need for mental health care among U.S. children: Variation by ethnicity and insurance status. The American Journal of Psychiatry, 159, 1548–1555.PubMedCrossRefGoogle Scholar
  26. Lambert, W., & Bickman, L. (2004). The “clock-setting” cure: How children’s symptoms might improve after ineffective treatment. Psychiatric Services, 55, 381–382.PubMedCrossRefGoogle Scholar
  27. Lipsey, M. W., & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48, 1181–1209.PubMedCrossRefGoogle Scholar
  28. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis (Vol. 49). Thousand Oaks, CA: Sage Publications.Google Scholar
  29. Liptzin, B. (2009). Quality improvement, pay for performance, and “outcomes measurement”: What makes sense? Psychiatric Services, 60, 108–111.PubMedCrossRefGoogle Scholar
  30. Manteuffel, B., Stephens, R. L., & Santiago, R. (2002). Overview of the national evaluation of the comprehensive community mental health services for children and their families program and summary of current findings. Children’s Services: Social Policy, Research, and Practice, 5, 3–20.CrossRefGoogle Scholar
  31. Manteuffel, B., Stephens, R. L., Sondheimer, D. L., & Fisher, S. K. (2008). Characteristics, service experiences, and outcomes of transition-aged youth in systems of care: Programmatic and policy implications. The Journal of Behavioral Health Services Research, 35, 469–487.PubMedCrossRefGoogle Scholar
  32. Maramba, G. G., Nagayama, H., & Gordon, C. (2002). Meta-analysis of ethnic match as a predictor of dropout, utilization, and level of functioning. Cultural Diversity and Ethnic Minority Psychology, 8, 290–297.PubMedCrossRefGoogle Scholar
  33. Maxwell, S. E., & Delaney, H. D. (2000). Designs with Concomitant Variables: ANCOVA and Blocking. In S. Maxwell & H. Delaney (Eds.), Designing Experiments and Analyzing Data. A Model Comparison Perspective (p. 256). Mahwah, New Jersey: Lawrence Erlbaum Associates.Google Scholar
  34. Ogles, B. M., Carlson, B., Hatfield, D., & Karpenko, V. (2008). Models of case mix adjustment for Ohio mental health consumer outcomes among children and adolescents. Administration and Policy in Mental Health and Mental Health Services Research, 35, 295–304.PubMedCrossRefGoogle Scholar
  35. Robinson, E. A., Eyberg, S. M., & Ross, W. A. (1980). The standardization of an inventory of child conduct problem behaviors. Journal of Clinical Child and Adolescent Psychology, 9, 22–28.CrossRefGoogle Scholar
  36. Rosenthal, R. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage Publications.Google Scholar
  37. Sederer, L. I., Hermann, R., & Dickey, B. (1995). The imperative of outcome assessment in Psychiatry. American Journal of Medical Quality, 10, 127–132.PubMedCrossRefGoogle Scholar
  38. Snowden, L. R. (2003). Bias in mental health assessment and intervention: Theory and evidence. American Journal of Public Health, 93, 239–243.PubMedCrossRefGoogle Scholar
  39. Soni, A. (2009, July). The five most costly conditions, 1996 and 2006: Estimates for the U.S. civilian noninstitutionalized population (Statistical Brief No. 248). Retrieved from Agency for Healthcare Research and Quality website: http://www.meps.ahrq.gov/mepsweb/data_files/publications/st248/stat248.pdf.
  40. Sourander, A., Helstela, L., & Helenius, H. (1999). Parent-adolescent agreement on emotional and behavioral problems. Social Psychiatry and Psychiatric Epidemiology, 34, 657–663.PubMedCrossRefGoogle Scholar
  41. Turchik, J. A., Karpenko, V., & Ogles, B. M. (2007). Further evidence of the utility and validity of a measure of outcome for children and adolescents. Journal of Emotional and Behavioral Disorders, 15, 119–128.CrossRefGoogle Scholar
  42. U.S. Department of Health and Human Services. (1999). Mental Health: A report of the Surgeon General. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National institute of Mental Health.Google Scholar
  43. Walrath, C., Garraza, L. G., Stephens, R., Azur, M., Miech, R., & Leaf, P. (2009). Trends in characteristics of children served by the Children’s Mental Health Initiative: 1994–2007. Administration and Policy in Mental Health and Mental Health Services Research, 36, 361–373.PubMedCrossRefGoogle Scholar
  44. Warren, J. S., Nelson, P. L., Mondragon, S. A., Baldwin, S. A., & Burlingame, G. M. (2010). Youth psychotherapy change trajectories and outcomes in usual care: Community mental health versus managed care settings. Journal of Consulting and Clinical Psychology, 78, 144–155.PubMedCrossRefGoogle Scholar
  45. Weisz, J. R. (2004). Psychotherapy for children and adolescents: Evidence-based treatments and case examples. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
  46. Weisz, J. R., & Jensen, A. L. (2001). Child and adolescent psychotherapy in research and practice contexts: Review of the evidence and suggestions for improving the field. European Journal of Child and Adolescent Psychiatry, 10, 12–18.CrossRefGoogle Scholar
  47. Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional psychology, research and practice, 24, 190–195.CrossRefGoogle Scholar
  48. Zima, B. T., Hurlburt, M. S., Knapp, P., Ladd, H., Tang, L., et al. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 130–144.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Department of Psychiatry, Child & Adolescent Services Research Center at Rady Children’s Hospital-San DiegoUniversity of CaliforniaSan DiegoUSA

Personalised recommendations