Skip to main content

Advertisement

SpringerLink
Log in

Cross-National Reliability of Clinician-Rated Outcome Measures in Child and Adolescent Mental Health Services

  • Original Paper
  • Published:
Administration and Policy in Mental Health and Mental Health Services Research Aims and scope Submit manuscript

Abstract

Clinician-rated measures are in extensive use as routine outcome measures in child and adolescent mental health services. We investigated cross-national differences and inter-rater reliability of the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), the Children’s Global Assessment Scale (CGAS) and the Global Assessment of Psychosocial Disability (GAPD). Thirty clinicians from 5 nations independently rated 20 written vignettes. The national groups afterwards established national consensus ratings. There were no cross-national differences in independent scores, but there were differences in national consensus scores, which were also more severe than independent scores. The ICC for the HoNOSCA total score was 0.84, for the CGAS 0.61 and for the GAPD 0.54. These measures may usefully contribute to cross-national comparison studies.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Becker, T., Knapp, M., Knudsen, H. C., Schene, A., Tansella, M., Thornicroft, G., et al. (1999). The EPSILON study of schizophrenia in five European countries. Design and methodology for standardising outcome measures and comparing patterns of care and service costs. British Journal of Psychiatry, 175, 514–521.

    CAS  PubMed  Google Scholar 

  • Bilenberg, N. (2003). Health of the nation outcome scales for children and adolescents (HoNOSCA). Results of a Danish field trial. European Child & Adolescent Psychiatry, 12(6), 298–302.

    Article  Google Scholar 

  • Brann, P., Coleman, G., & Luk, E. (2001). Routine outcome measurement in a child and adolescent mental health service: an evaluation of HoNOSCA. The Australian and New Zealand Journal of Psychiatry, 35(3), 370–376.

    Article  CAS  PubMed  Google Scholar 

  • Canino, G., & Bravo, M. (1999). The translation and adaptation of diagnostic instruments for cross-cultural use. In D. Shaffer, C. P. Lucas, & J. E. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp. 285–298). New York: The Guilford Press.

    Google Scholar 

  • Department of Health and Ageing (2003). Mental health national outcomes and casemix collection: Technical specification of state and territory reporting requirements for the outcomes and casemix components of ‘agreed data’, Version 1.50. Canberra, Australia: Commonwealth Government.

    Google Scholar 

  • Dopfner, M., Steinhausen, H. C., Coghill, D., Dalsgaard, S., Poole, L., Ralston, S. J., et al. (2006). Cross-cultural reliability and validity of ADHD assessed by the ADHD Rating Scale in a pan-European study. European Child & Adolescent Psychiatry, 15(Suppl 1), i46–i55.

    Article  Google Scholar 

  • Dyrborg, J., Larsen, F. W., Nielsen, S., Byman, J., Nielsen, B. B., & Gautre-Delay, F. (2000). The Children’s Global Assessment Scale (CGAS) and Global Assessment of Psychosocial Disability (GAPD) in clinical practice - substance and reliability as judged by intraclass correlations. European Child & Adolescent Psychiatry, 9(3), 195–201.

    Article  CAS  Google Scholar 

  • Efron, B., & Tibshirani, R. J. (1993). An introduction to the Bootstrap. Chapman & Hall.

  • Garralda, M. E., Yates, P., & Higginson, I. (2000). Child and adolescent mental health service use. HoNOSCA as an outcome measure. British Journal of Psychiatry, 177, 52–58.

    Article  CAS  PubMed  Google Scholar 

  • Gowers, S. G., Harrington, R. C., Whitton, A., Lelliott, P., Beevor, A., Wing, J., et al. (1999). Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for children and Adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413–416.

    CAS  PubMed  Google Scholar 

  • Johnston, C., & Gowers, S. (2005). Routine Outcome Measurement: A Survey of UK Child and Adolescent Mental Health Services. Child and Adolescent Mental Health, 10(3), 133–139.

    Article  Google Scholar 

  • MH-SMART (15.2.2007). The Mental Health Standard Measures of Assessment and Recovery Summary. http://www.tepou.co.nz/page/tepou_11.php. Access date 19.3.2007.

  • Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., et al. (2004). Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Annals of Internal Medicine, 141(10), 771–780.

    PubMed  Google Scholar 

  • Rey, J. M., Starling, J., Wever, C., Dossetor, D. R., & Plapp, J. M. (1995). Inter-rater reliability of global assessment of functioning in a clinical setting. Journal of Child Psychology and Psychiatry, 36(5), 787–792.

    Article  CAS  PubMed  Google Scholar 

  • Schorre, B. E. & Vandvik, I. H. (2004). Global assessment of psychosocial functioning in child and adolescent psychiatry. A review of three unidimensional scales (CGAS, GAF, GAPD). European Child & Adolescent Psychiatry, 13(5), 273–286.

    Article  Google Scholar 

  • Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228–1231.

    CAS  PubMed  Google Scholar 

  • Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317.

    Article  CAS  PubMed  Google Scholar 

  • Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.

    Article  Google Scholar 

  • Weiss, B. (1998). Annotation: Routine monitoring of the effectiveness of child psychotherapy. Journal of Child Psychology and Psychiatry, 39(7), 943–950.

    Article  CAS  PubMed  Google Scholar 

  • World Health Organization (1996). Multiaxial classification of child and adolescent psychiatric disorders. Cambridge University Press.

  • Yates, P., Garralda, M. E., & Higginson, I. (1999). Paddington complexity scale and health of the nation outcome scales for children and adolescents. British Journal of Psychiatry, 174, 417–423.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Thanks to the raters who rated the vignettes and discussed the ratings until consensus was achieved: Peter Birleson, Jens Buchhave, Tania Cargo, Merrie Carling, Tim Coombs, Per-Erik Davidsen, Heather Mc Dowell, Berit Groholt, Kirsten Hoerder, Jenny Hoffman, Timothy Hughes, Torben Isager, Craig Johnston, Margaret Jones, Sarah Laing, Flemming Warborg Larsen, Michael Maagensen, Torunn S Novik, Epenesa Olo-Whaanga, Andy Parkin, Peter Roots, Benedicte Skirbekk, Su Sukamaran and Sue Treanor. The study was supported by The Research Council of Norway and by Centre for Child and Adolescent Mental Health, Eastern and Southern Norway. Disclosure: The authors report no conflicts of interests.

Author information

Authors and Affiliations

  1. Centre for Child and Adolescent Mental Health, Eastern and Southern Norway, P.O. Box 4623, Nydalen, Oslo, NO-0405, Norway

    Ketil Hanssen-Bauer & Sonja Heyerdahl

  2. Section of Adolescent Psychiatry, University of Liverpool, Chester, UK

    Simon Gowers

  3. Department of Biostatistics, Institute of Basic Medical Sciences, University of Oslo, Oslo, Norway

    Odd O. Aalen

  4. Child and Adolescent Psychiatric Department, University of Southern Denmark, Odense, Denmark

    Niels Bilenberg

  5. Eastern Health Child and Adolescent Mental Health Service, Victoria, Australia

    Peter Brann

  6. Academic Unit of Child and Adolescent Psychiatry, Imperial College London, London, UK

    Elena Garralda

  7. Werry Centre for Child and Adolescent Mental Health, Department of Psychological Medicine, University of Auckland, Auckland, New Zealand

    Sally Merry

  8. Department of Psychological Medicine, Monash University, Victoria, Australia

    Peter Brann

Authors
  1. Ketil Hanssen-Bauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Gowers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Odd O. Aalen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Niels Bilenberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Brann
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Elena Garralda
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sally Merry
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sonja Heyerdahl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ketil Hanssen-Bauer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hanssen-Bauer, K., Gowers, S., Aalen, O.O. et al. Cross-National Reliability of Clinician-Rated Outcome Measures in Child and Adolescent Mental Health Services. Adm Policy Ment Health 34, 513–518 (2007). https://doi.org/10.1007/s10488-007-0135-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10488-007-0135-x

Keywords

Search

Navigation