Cross-National Reliability of Clinician-Rated Outcome Measures in Child and Adolescent Mental Health Services

  • Ketil Hanssen-BauerEmail author
  • Simon Gowers
  • Odd O. Aalen
  • Niels Bilenberg
  • Peter Brann
  • Elena Garralda
  • Sally Merry
  • Sonja Heyerdahl
Original Paper


Clinician-rated measures are in extensive use as routine outcome measures in child and adolescent mental health services. We investigated cross-national differences and inter-rater reliability of the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA), the Children’s Global Assessment Scale (CGAS) and the Global Assessment of Psychosocial Disability (GAPD). Thirty clinicians from 5 nations independently rated 20 written vignettes. The national groups afterwards established national consensus ratings. There were no cross-national differences in independent scores, but there were differences in national consensus scores, which were also more severe than independent scores. The ICC for the HoNOSCA total score was 0.84, for the CGAS 0.61 and for the GAPD 0.54. These measures may usefully contribute to cross-national comparison studies.


Children Adolescent Mental health Services Reliability Outcome 



Thanks to the raters who rated the vignettes and discussed the ratings until consensus was achieved: Peter Birleson, Jens Buchhave, Tania Cargo, Merrie Carling, Tim Coombs, Per-Erik Davidsen, Heather Mc Dowell, Berit Groholt, Kirsten Hoerder, Jenny Hoffman, Timothy Hughes, Torben Isager, Craig Johnston, Margaret Jones, Sarah Laing, Flemming Warborg Larsen, Michael Maagensen, Torunn S Novik, Epenesa Olo-Whaanga, Andy Parkin, Peter Roots, Benedicte Skirbekk, Su Sukamaran and Sue Treanor. The study was supported by The Research Council of Norway and by Centre for Child and Adolescent Mental Health, Eastern and Southern Norway. Disclosure: The authors report no conflicts of interests.


  1. Becker, T., Knapp, M., Knudsen, H. C., Schene, A., Tansella, M., Thornicroft, G., et al. (1999). The EPSILON study of schizophrenia in five European countries. Design and methodology for standardising outcome measures and comparing patterns of care and service costs. British Journal of Psychiatry, 175, 514–521.PubMedGoogle Scholar
  2. Bilenberg, N. (2003). Health of the nation outcome scales for children and adolescents (HoNOSCA). Results of a Danish field trial. European Child & Adolescent Psychiatry, 12(6), 298–302.CrossRefGoogle Scholar
  3. Brann, P., Coleman, G., & Luk, E. (2001). Routine outcome measurement in a child and adolescent mental health service: an evaluation of HoNOSCA. The Australian and New Zealand Journal of Psychiatry, 35(3), 370–376.CrossRefPubMedGoogle Scholar
  4. Canino, G., & Bravo, M. (1999). The translation and adaptation of diagnostic instruments for cross-cultural use. In D. Shaffer, C. P. Lucas, & J. E. Richters (Eds.), Diagnostic assessment in child and adolescent psychopathology (pp. 285–298). New York: The Guilford Press.Google Scholar
  5. Department of Health and Ageing (2003). Mental health national outcomes and casemix collection: Technical specification of state and territory reporting requirements for the outcomes and casemix components of ‘agreed data’, Version 1.50. Canberra, Australia: Commonwealth Government.Google Scholar
  6. Dopfner, M., Steinhausen, H. C., Coghill, D., Dalsgaard, S., Poole, L., Ralston, S. J., et al. (2006). Cross-cultural reliability and validity of ADHD assessed by the ADHD Rating Scale in a pan-European study. European Child & Adolescent Psychiatry, 15(Suppl 1), i46–i55.CrossRefGoogle Scholar
  7. Dyrborg, J., Larsen, F. W., Nielsen, S., Byman, J., Nielsen, B. B., & Gautre-Delay, F. (2000). The Children’s Global Assessment Scale (CGAS) and Global Assessment of Psychosocial Disability (GAPD) in clinical practice - substance and reliability as judged by intraclass correlations. European Child & Adolescent Psychiatry, 9(3), 195–201.CrossRefGoogle Scholar
  8. Efron, B., & Tibshirani, R. J. (1993). An introduction to the Bootstrap. Chapman & Hall.Google Scholar
  9. Garralda, M. E., Yates, P., & Higginson, I. (2000). Child and adolescent mental health service use. HoNOSCA as an outcome measure. British Journal of Psychiatry, 177, 52–58.CrossRefPubMedGoogle Scholar
  10. Gowers, S. G., Harrington, R. C., Whitton, A., Lelliott, P., Beevor, A., Wing, J., et al. (1999). Brief scale for measuring the outcomes of emotional and behavioural disorders in children. Health of the Nation Outcome Scales for children and Adolescents (HoNOSCA). British Journal of Psychiatry, 174, 413–416.PubMedGoogle Scholar
  11. Johnston, C., & Gowers, S. (2005). Routine Outcome Measurement: A Survey of UK Child and Adolescent Mental Health Services. Child and Adolescent Mental Health, 10(3), 133–139.CrossRefGoogle Scholar
  12. MH-SMART (15.2.2007). The Mental Health Standard Measures of Assessment and Recovery Summary. Access date 19.3.2007.
  13. Peabody, J. W., Luck, J., Glassman, P., Jain, S., Hansen, J., Spell, M., et al. (2004). Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Annals of Internal Medicine, 141(10), 771–780.PubMedGoogle Scholar
  14. Rey, J. M., Starling, J., Wever, C., Dossetor, D. R., & Plapp, J. M. (1995). Inter-rater reliability of global assessment of functioning in a clinical setting. Journal of Child Psychology and Psychiatry, 36(5), 787–792.CrossRefPubMedGoogle Scholar
  15. Schorre, B. E. & Vandvik, I. H. (2004). Global assessment of psychosocial functioning in child and adolescent psychiatry. A review of three unidimensional scales (CGAS, GAF, GAPD). European Child & Adolescent Psychiatry, 13(5), 273–286.CrossRefGoogle Scholar
  16. Shaffer, D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., et al. (1983). A children’s global assessment scale (CGAS). Archives of General Psychiatry, 40(11), 1228–1231.PubMedGoogle Scholar
  17. Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317.CrossRefPubMedGoogle Scholar
  18. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.CrossRefGoogle Scholar
  19. Weiss, B. (1998). Annotation: Routine monitoring of the effectiveness of child psychotherapy. Journal of Child Psychology and Psychiatry, 39(7), 943–950.CrossRefPubMedGoogle Scholar
  20. World Health Organization (1996). Multiaxial classification of child and adolescent psychiatric disorders. Cambridge University Press.Google Scholar
  21. Yates, P., Garralda, M. E., & Higginson, I. (1999). Paddington complexity scale and health of the nation outcome scales for children and adolescents. British Journal of Psychiatry, 174, 417–423.PubMedCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Ketil Hanssen-Bauer
    • 1
    Email author
  • Simon Gowers
    • 2
  • Odd O. Aalen
    • 3
  • Niels Bilenberg
    • 4
  • Peter Brann
    • 5
    • 8
  • Elena Garralda
    • 6
  • Sally Merry
    • 7
  • Sonja Heyerdahl
    • 1
  1. 1.Centre for Child and Adolescent Mental Health, Eastern and Southern NorwayOsloNorway
  2. 2.Section of Adolescent PsychiatryUniversity of LiverpoolChesterUK
  3. 3.Department of Biostatistics, Institute of Basic Medical SciencesUniversity of OsloOsloNorway
  4. 4.Child and Adolescent Psychiatric DepartmentUniversity of Southern DenmarkOdenseDenmark
  5. 5.Eastern Health Child and Adolescent Mental Health ServiceVictoriaAustralia
  6. 6.Academic Unit of Child and Adolescent PsychiatryImperial College LondonLondonUK
  7. 7.Werry Centre for Child and Adolescent Mental Health, Department of Psychological MedicineUniversity of AucklandAucklandNew Zealand
  8. 8.Department of Psychological MedicineMonash UniversityVictoriaAustralia

Personalised recommendations