Selecting Measures for Use in Child Mental Health Services: A Scorecard Approach

  • Erin M. Warnick
  • V. Robin Weersing
  • Lawrence Scahill
  • Joseph L. Woolston
Original paper

Abstract

The implementation of evidence-based treatments in mental health services requires reliable and valid measurements to guide treatment. This study evaluated the efficiency of three caretaker-report measures of child psychiatric disorders. Data from 211 caregivers were used to assess the CBCL, the OHIO, and the SDQ. A scorecard methodology was implemented to determine the efficiency of each scale as compared to the DISC-IV. Across measures, the OHIO was optimal for assessing services need while the CBCL and SDQ provided better disorder-specific assessment. Results may improve clinical practice by providing an empirical approach to the selection of assessment tools.

Keywords

Child mental health Diagnosis Rating scales Diagnostic efficiency 

References

  1. Achenbach, T. M., & Rescorla, L. (2001). Manual for the ASEBA school-age forms and profiles: An integrated system of multi-informant assessment. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families.Google Scholar
  2. Achenbach system of empirically based assessment (ASEBA). (2008). 2008 ASEBA Catalog. Retrieved November 25, 2008, from http://www.aseba.org/2008%20Catalog.pdf.
  3. Angold, A., Costello, E. J., & Erkanli, A. (1999a). Comorbidity. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 40(1), 57–87. doi: 10.1111/1469-7610.00424.PubMedCrossRefGoogle Scholar
  4. Angold, A., Costello, E. J., Farmer, E. M. Z., Burns, B. J., & Erkanli, A. (1999b). Impaired but undiagnosed. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 129–137. doi: 10.1097/00004583-199902000-00011.PubMedCrossRefGoogle Scholar
  5. Baize, H. (2001). Implications of the Ohio scales factor structure for scale utility and scoring. San Francisco, CA: The Fourth Annual Californian Children’s System of Care Model Evaluation Conference.Google Scholar
  6. Birmaher, B., Brent, D. A., Chiappetta, L., Bridge, J., Monga, S., & Baugher, M. (1999). Psychometric properties of the screen for child anxiety related emotional disorders (SCARED): A replication study. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1230–1236. doi: 10.1097/00004583-199910000-00011.PubMedCrossRefGoogle Scholar
  7. CDECP Research section. (2000). Connecticut Town Profiles 1998–1999. New Haven, CT: Connecticut Department of Economic and Community Development. Retrieved February 28, 2007, from www.state.ct.us/ecd/research.
  8. Chen, W. J., Faraone, S. V., Biederman, J., & Tsuang, M. T. (1994). Diagnostic accuracy of the child behavior checklist scales for attention-deficit hyperactivity disorder: A receiver-operating characteristic analysis. Journal of Consulting and Clinical Psychology, 62(5), 1017–1025. doi: 10.1037/0022-006X.62.5.1017.PubMedCrossRefGoogle Scholar
  9. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. doi: 10.1177/001316446002000104.CrossRefGoogle Scholar
  10. Costello, E., Mustillo, S., Erkanli, A., Keeler, G., & Angold, A. (2003). Prevalence and development of psychiatric disorders in childhood and adolescence. Archives of General Psychiatry, 60, 837–844. doi: 10.1001/archpsyc.60.8.837.PubMedCrossRefGoogle Scholar
  11. Costello, E. J., & Shugart, M. A. (1992). Above and below the threshold: severity of psychiatric symptoms and functional impairment in a pediatric sample. Pediatrics, 90, 359–368.PubMedGoogle Scholar
  12. De Los Reyes, A., & Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: A critical review, theoretical framework, and recommendations for further study. Psychological Bulletin, 131, 483–509. doi: 10.1037/0033-2909.131.4.483.PubMedCrossRefGoogle Scholar
  13. Doss, A. J. (2005). Evidence-based diagnosis: Incorporating diagnostic instruments into clinical practice. Journal of the American Academy of Child and Adolescent Psychiatry, 44, 947–952. doi: 10.1097/01.chi.0000171903.16323.92.PubMedCrossRefGoogle Scholar
  14. Epkins, C. C. (2002). A comparison of two self-report measures of children’s social anxiety in clinic and community samples. Journal of Clinical Child and Adolescent Psychology, 31, 69–79.PubMedGoogle Scholar
  15. First, M., Gibbon, M., Spitzer, R. L., & Williams, J. B. (2004). What is the Validity of the SCID-I. Retrieved December 8, 2004, from http://www.scid4.org.
  16. Forbes, G. B. (2001). A comparison of the Conner’s parent and teacher rating scales, the add-h comprehensive teacher’s rating scale and the child behavior checklist in the clinical diagnosis of ADHD. Journal of Attention Disorders, 5, 25–40. doi: 10.1177/108705470100500103.CrossRefGoogle Scholar
  17. Goodman, R. (1997). The strengths and difficulties questionnaire: A research note. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 38, 581–586. doi: 10.1111/j.1469-7610.1997.tb01545.x.PubMedCrossRefGoogle Scholar
  18. Goodman, R. (1999). The extended version of the strengths and difficulties questionnaire as a guide to child psychiatric caseness and consequent burden. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 40, 791–799. doi: 10.1111/1469-7610.00494.PubMedCrossRefGoogle Scholar
  19. Goodman, R. (2001). Psychometric properties of the strengths and difficulties questionnaire. Journal of the American Academy of Child and Adolescent Psychiatry, 40, 1337–1345. doi: 10.1097/00004583-200111000-00015.PubMedCrossRefGoogle Scholar
  20. Goodman, R., & Scott, S. (1999). Comparing the strengths and difficulties questionnaire and the child behavior checklist: Is small beautiful? Journal of Abnormal Child Psychology, 27, 17–24. doi: 10.1023/A:1022658222914.PubMedCrossRefGoogle Scholar
  21. Hawes, D. J., & Dadds, M. R. (2004). Australian data and psychometric properties of the strengths and difficulties questionnaire. The Australian and New Zealand Journal of Psychiatry, 38, 644–651. doi: 10.1111/j.1440-1614.2004.01427.x.PubMedCrossRefGoogle Scholar
  22. Hoagwood, K., Jensen, P. S., Petti, T., & Burns, B. J. (1996). Outcomes of mental health care for children and adolescents: I. A comprehensive conceptual model. Journal of the American Academy of Child and Adolescent Psychiatry, 35(8), 1055–1063. doi: 10.1097/00004583-199608000-00017.PubMedCrossRefGoogle Scholar
  23. Hodges, K., & Wong, M. M. (1996). Psychometric characteristics of a multidimensional measure to assess impairment: The child and adolescent functional assessment scale. Journal of Child and Family Studies, 5(4), 445–467. doi: 10.1007/BF02233865.CrossRefGoogle Scholar
  24. Hudziak, J. J., Copeland, W., Stanger, C., & Wadsworth, M. (2004). Screening for DSM-IV externalizing disorders with the child behavior checklist: A receiver-operating characteristic analysis. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 45(7), 1299–1307. doi: 10.1111/j.1469-7610.2004.00314.x.PubMedCrossRefGoogle Scholar
  25. Jensen, P. S., Salzberg, A. D., Richters, J. E., & Watanabe, H. K. (1993). Scales, diagnoses, and child psychopathology: I. CBCL and DISC relationships. Journal of the American Academy of Child and Adolescent Psychiatry, 32, 397–406.PubMedGoogle Scholar
  26. Jensen, P. S., Rubio-Stipec, M., Canino, G., Bird, H. R., Dulcan, M. K., Schwab-Stone, M. E., et al. (1999a). Parent and child contributions to diagnosis of mental disorder: Are both informants always necessary? Journal of the American Academy of Child and Adolescent Psychiatry, 38, 1569–1579.PubMedCrossRefGoogle Scholar
  27. Jensen, P., Watanabe, H., & Richters, J. (1999b). Who’s up first? Testing for order effects in structured interviews using a counterbalanced experimental design. Journal of Abnormal Child Psychology, 27(6), 439–445. doi: 10.1023/A:1021927909027.PubMedCrossRefGoogle Scholar
  28. Jensen, A. L., & Weisz, J. R. (2002). Assessing match and mismatch between practitioner-generated and standardized interview-generated diagnoses for clinic-referred children and adolescents. Journal of Consulting and Clinical Psychology, 70, 158–168. doi: 10.1037/0022-006X.70.1.158.PubMedCrossRefGoogle Scholar
  29. Jensen, P. S., Hoagwood, K., & Zitner, L. (Eds.). (2006). What’s in a name? Problems versus prospects in current diagnostic approaches. Hoboken, NJ, USA: Wiley.Google Scholar
  30. Klasen, H., Woerner, W., Wolke, D., Meyer, R., Overmeyer, S., Kaschnitz, W., et al. (2000). Comparing the German versions of the strengths and difficulties questionnaire (SDQ-Deu) and the child behavior checklist. European Child & Adolescent Psychiatry, 9, 271–276. doi: 10.1007/s007870070030.CrossRefGoogle Scholar
  31. Noterdaeme, M., & Amorosa, H. (1999). Evaluation of emotional and behavioral problems in language impaired children using the child behavior checklist. European Child & Adolescent Psychiatry, 8, 71–77. doi: 10.1007/s007870050087.CrossRefGoogle Scholar
  32. Ogles, B. M., Melendez, G., Davis, D. C., & Lunnen, K. M. (2000). The Ohio youth problem, functioning, and satisfaction scales: Technical manual. In: Ohio Mental health consumer outcomes initiative Ohio: Ohio University.Google Scholar
  33. Ohio Department of Mental Health. (2005). Instruments and procedures: Outcomes instruments, English versions. Retrieved November 25, 2008, from http://www.mh.state.oh.us/oper/outcomes/instruments.english.html.
  34. USA Public Health Service. (2000). Report of the surgeon general’s conference on children’s mental health: A national action agenda. Washington, DC: Department of Health and Human Services.Google Scholar
  35. Ramirez-Basco, M., Bostic, J. Q., Davies, D., Rush, A., Witte, B., Hendrickse, W., et al. (2000). Methods to improve diagnostic accuracy in a community mental health setting. The American Journal of Psychiatry, 157, 1599–1605. doi: 10.1176/appi.ajp.157.10.1599.PubMedCrossRefGoogle Scholar
  36. Roberts, N., Vargo, B., & Ferguson, H. B. (1989). Measurement of anxiety and depression in children and adolescents. The Psychiatric Clinics of North America, 12, 837–860.PubMedGoogle Scholar
  37. Rubio-Stipec, M., Fitzmaurice, G., Murphy, J., & Walker, A. (2003). The use of multiple informants in identifying the risk factors of depressive and disruptive disorders. Are they interchangeable? Social Psychiatry and Psychiatric Epidemiology, 38, 51–58. doi: 10.1007/s00127-003-0600-0.PubMedCrossRefGoogle Scholar
  38. Rubio-Stipec, M., Walker, A., Murphy, J., & Fitzmaurice, G. (2002). Dimensional measures of psychopathology. The probability of being classified with a psychiatric disorder using empirically derived symptom scales. Social Psychiatry and Psychiatric Epidemiology, 37, 553–560. doi: 10.1007/s00127-002-0561-8.PubMedCrossRefGoogle Scholar
  39. SAMHSA. (2003). National mental health information center: Child and adolescent mental health. USA Department of Health and Human Services.Google Scholar
  40. SAS Institute Inc. (2003). SAS 9.1 (9.1st ed.). NC: SAS Institute, Inc.Google Scholar
  41. Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M. K., & Schwab-Stone, M. E. (2000). NIMH diagnostic interview schedule for children, version IV (NIMH DISC-IV): Description, differences from previous versions, and reliability of some common diagnoses. Journal of the American Academy of Child and Adolescent Psychiatry, 39, 28–38. doi: 10.1097/00004583-200001000-00014.PubMedCrossRefGoogle Scholar
  42. SPSS. (2004). SPSS 13.0 for Windows. 13.0.1 Edition: LEAD Technologies, Inc.Google Scholar
  43. Turchik, J. A., Karpenko, V., & Ogles, B. M. (2007). Further evidence of the utility and validity of a measure of outcome for children and adolescents. Journal of Emotional and Behavioral Disorders, 15(2), 119–128. doi: 10.1177/10634266070150020201.CrossRefGoogle Scholar
  44. Warnick, E. M., Bracken, M. B., & Kasl, S. (2008). Screening efficiency of the child behavior checklist and strengths and difficulties questionnaire: A systematic review. Child and Adolescent Mental Health, 13(3), 140–147. doi: 10.1111/j.1475-3588.2007.00461.x.CrossRefGoogle Scholar
  45. Youth in Mind (2005). SDQ: Information for researchers and professionals about the Strengths & Difficulties Questionnaires. Retrieved August 22, 2005, from http://www.sdqinfo.com/.

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Erin M. Warnick
    • 1
  • V. Robin Weersing
    • 1
    • 2
  • Lawrence Scahill
    • 1
    • 3
  • Joseph L. Woolston
    • 1
    • 4
  1. 1.Yale University Child Study CenterNew HavenUSA
  2. 2.SDSU/UCSD Joint Doctoral Program in Clinical PsychologySan DiegoUSA
  3. 3.Yale University Child Study CenterYale University School of NursingNew HavenUSA
  4. 4.Yale University Child Study CenterNew HavenUSA

Personalised recommendations