The aim of this commentary is to provide an overview of clinical outcome measures that are currently recommended for use in UK Child and Adolescent Mental Health Services (CAMHS), focusing on measures that are applicable across a wide range of conditions with established validity and reliability, or innovative in their design. We also provide an overview of the barriers and drivers to the use of Routine Outcome Measurement (ROM) in clinical practice.
For the purpose of this paper, we define ROM as the use of generic measures that assess the clinical outcomes or patient/carer satisfaction with service delivery. Outcome measures are usually completed at first contact (baseline) and after a fixed interval, often 6 months after the initial measure . Symptomatic measures or measures of broader functioning that are completed only at one time point (e.g. at the end of intervention) do not provide a measure of within-individual change which is an essential feature of symptomatic or functional outcome measurement. The exception to this rule is measures of patient or caregiver satisfaction with the service which are typically obtained once at the end of treatment or discharge [2, 3].
The purpose and use of outcome measures may differ depending upon the end user of the data. ROM should enable clinicians to assess change over the course of treatment and help them draw comparisons between the perspectives of the clinician, child, their parent/carer and other informants such as teachers [4, 5]. Outcome measures provide service users with a way of seeing change in their condition and functioning over time and an opportunity to express their level of satisfaction with the care received . At a service level, outcome data can help identify areas for development, evaluate whether services are meeting targets and influence the allocation of funding . Anonymised outcome data collected at a service level may satisfy commissioners’ demand for greater service accountability through service user feedback and objective measurement of clinical effectiveness . Regular, consistent outcome measurement should lead to improvements in practice and patient outcome, provided that results are carefully interpreted in the clinical and organisational context .
Fitzpatrick and colleagues  outline several criteria that outcome measures should meet. ROM should be based on measures with good psychometric properties, including established reliability, validity and sensitivity to change. Measures should be simple and quick to complete, cost-effective and easy to interpret. Furthermore, if outcome measures are to be used for benchmarking, they should be generic, relevant to the most frequent clinical diagnoses and applicable across a broad range of theoretical frameworks. Generic outcome measures do not cover factors specific to all disorders, but enable comparisons across disorders and services. Outcome measures data should be interpreted in the context of case mix and case complexity for each particular service.
The NHS National Service Framework for Children, Young People and Maternity Services  in England proposed that work conducted within CAMHS should be evaluated from the perspective of both clinicians and service users. In response to this, the CAMHS Outcome Research Consortium (CORC; ) was created to develop a common suite of measures and to provide leadership on CAMHS ROM and support to services with the collection and analysis of anonymised outcome data. CORC recommend a range of core outcome measures , including: the Strengths and Difficulties Questionnaire (SDQ) ; the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA)  and the Children’s Global Assessment Scale (C-GAS) . These measures were chosen for their established validity, reliability and applicability across a range of psychiatric problems in children and young people [10, 14] and open source access free of charge. CORC also recommend the use of two more recently developed measures whose psychometric properties are less well known: the Commission for Health Improvement-Experience of Service Questionnaire (CHI-ESQ)  as a measure of service user satisfaction, and the Goals Based Outcome (GBO) . However, CORC do not advocate that ROM should be limited only to their selected measures .
Since 2011, CORC have been commissioned by the Department of Health to support the analysis of outcome measurements collated through the Children and Young People’s Improving Access to Psychological Therapies (CYP-IAPT; www.IAPT.nhs.uk). The CYP-IAPT aspires to improve services for service users by routinely assessing their opinion on the quality and experience of services. Alongside the SDQ, GBO and CHI-ESQ, CYP-IAPT recommend the use of brief scales such as the ORS (Outcome Rating Scale)  to measure functioning and the SRS (Session Rating Scale)  to assess client satisfaction on a session-by-session basis (http://www.iapt.nhs.uk/silo/files/cyp-iapt-outcomes-summary.pdf). Although there has been some research on the psychometric properties of the adult versions of these scales [18, 19], there is no research investigating the psychometric properties of these child versions.
Drawing on the work of Clark et al.  and Weiz et al.  CYP-IAPT specifically advocates the use of idiographic and standardised measures. In their commentary, Wolpert et al.  specifically make reference to the compromise of choosing measures which are sufficiently tailored to individual patient needs to be able to inform clinical practice whilst being broad enough to draw comparisons across cases and services. They also comment that the CYP-IAPT measures have been chosen with recognition of the need to reduce time burden for both the clinical staff and service users whilst balancing reliability and generalisability.
Barriers to the implementation of ROM into clinical practice
The literature reveals common themes that are recognised barriers and facilitators to the implementation of ROM in mental health services internationally.
Studies have shown that treatment outcomes are measured in only 16–30 % of clinical cases [2, 3] in the UK, and only 37 % of psychologists in the US  reported measuring outcomes routinely; these findings suggest the presence of barriers to ROM in practice. Several studies have identified multiple barriers, including the increased time demands on clinicians and administrative staff [2, 3, 17, 22, 24]; a lack of clinician training on how to integrate ROM into clinical practice ; whether ROM is considered to produce clinically useful information [19, 23, 25, 26]; and a poor return rate of questionnaires completed by service users outside the clinic [27–29]. The frequent lack of timely feedback from outcome measures decreases their clinical usefulness and has a negative impact on clinicians’ and patients’ motivation to use them .
Additional barriers are related to limitations of the available outcome measures, for example, the fact that generic outcome measures do not typically assess self-harm behaviour and suicidal risk , or are not sensitive to symptom change in some clinical presentations .
Clinicians have expressed concerns about the time required to complete the questionnaires during the session [2, 32] and about outcome measures not being necessary  or relevant to their practice [23, 34]. Differences in psychologists’ willingness to use outcome measures according to their therapeutic approach (i.e. cognitive behavioural or insight-oriented) have been reported , and some clinicians may be reluctant to use a quantitative, systematic approach for data collection . Johnston and Gowers  found that clinicians who did not regularly use quantitative clinical measurements were more likely to be sceptical about the value of the quantitative ‘medical’ approach. Additional concerns relate to ‘labelling’ patients , confidentiality , and the risk of data being used by managers and commissioners to unfairly compare services that deal with different levels of case complexity [23, 24, 32]. Despite these reservations, most studies [2, 24, 36] report a range of clinicians’ views, with a substantial number of clinicians showing a positive attitude towards the implementation of ROM. Clinicians’ attitudes towards ROM have been shown to become more positive following attendance at a workshop and training focusing on their clinical value .
Studies have reported  that parents feel that ROM can add to the burden of form-filling already required of service users, even when language is not a barrier ; however, positive views about the opportunity to express their opinion have also been reported .
In summary, research shows that clinicians and service users have a range of views about ROM and identifies the need for further clinician training on the use of outcome measures, as well as a system to improve the provision of timely feedback from those measures to clinicians and patients/carers to support real-time clinical decision-making.
Future directions for development include the validation of session-by-session outcome measures  and the use of technology (for example, computer-based measures) to aid ROM implementation and reduce administrative burden. Research into the barriers to the integration of ROM in CAMHS in different countries will be of international interest and may provide insights into methods that support wider uptake of ROM and further evidence for their contribution to improved clinical effectiveness of child and adolescent mental healthcare.
Ford T, Hutchings J, Bywater T, Goodman A, Goodman R (2009) Strengths and difficulties questionnaire added value scores: evaluating effectiveness in child mental health interventions. Br J Psychia 194:552–558
Batty M, Moldavsky M, Sarrami Foroushani P, Pass S, Marriott M, Sayal K, Hollis C (2012) Implementing routine outcome measures in child and adolescent mental health services: from present to future practice. Child Adol Ment Heal 18:82–87
Johnston C, Gowers S (2005) Routine outcome measurement: a survey of UK child and adolescent mental health services. Child Adolesc Ment Heal 10:133–139
Ford T, Tingay T, Wolpert M, Group CS (2006) CORC’s survey of routine outcome monitoring and national CAMHS dataset developments: a response to Johnston and Gower. Child Adolesc Ment Heal 11:50–52
Garralda ME, Yates P, Higginson I (2000) Child and adolescent mental health service use: HoNOSCA as an outcome measure. Br J Psychia 177:52–58
Barkham MF, Margison F, Leach C et al (2001)Service profiling and outcomes benchmarking using the CORE-OM: toward practice-based evidence in the psychological therapies. J Consult Clin Psychol 69:184–196
Fitzpatrick RC, Davey C, Buxton MJ, Jones DR (1998)Evaluating patient-based outcome measures for use in clinical trials. Heal Technol Assess (Winchester, England) 2:1–74
Department of Health (2007) National service framework for children, young people and maternity services: the mental health and psychological well-being of children and young people: standard-9, Department of Health, London
CAMHS Outcome Research Consortium (CORC): www.corc.uk.net. Accessed 1 June 2012
Wolpert M, Cooper L, Tingay K, Young K, Svanberg E (2007) CAMHS outcomes research consortium handbook: version 2.0. CORC, London
Goodman R (1997) The strengths and difficulties questionnaire: a research note. J Child Psychol Psychia All Discip 38:581–586
Gowers SG, Harrington RC, Whitton A (1999) Brief scale for measuring the outcomes of emotional and behavioural disorders in children––health of the nation outcome scales for children and adolescents (HoNOSCA). Br J Psychia 174:413–416
Shaffer D, Gould MS, Brasic J et al (1983) A children’s global assessment scale (CGAS). Arch Gen Psychia 40:1228–1238
Attride-Stirling J (2003) Development of methods to capture users’ views of child and adolescent mental health services in clinical governance reviews (Project evaluation report). http://www.corc.uk.net/wp.../CHI-evaluation-report-Attride-Stirling-J1.pdf. Accessed 26 July 2012
Law D (2011) Goal Based Outcomes (GBOs): Some useful information. http://www.corc.uk.net/resources/downloads. Accessed 21st July 2012
Johnson LD, Miller SD, Duncan BL (2000) The session rating scale 30: Chicago
Duncan BL, Miller SD, Sparks JA, Claud DA, Reynolds LR, Brown J, Johnson LD (2003) The session rating scale: preliminary psychometric properties of a “working” alliance measure. J Brief Ther 3:3–12
Miller SD, Duncan BL (2000) The outcome rating scale. Authors, Chicago
Miller SD, Duncan BL, Brown J, Sparks JA, Claud DA (2003) The outcome rating scale: a preliminary study of the reliability, validity, and feasibility of a brief visual analog measure. J Brief Ther 2:91–100
Clark SM, Fairburn CG, Wessely S (2008) Psychological treatment outcomes in routine NHS services: a commentary on Stiles et al. (2007). Psychol Med 38:634–829
Weiz JR, Chorpita BF, Fyre A, NG MY, Lau N, Bearman SK, Research Network on Youth Mental Health (2011) Youth top problems: using idiographic, consumer guided assessment to identify treatment needs and to track change during psychotherapy. J Cons Clin Psychia 29:369–380
Wolpert M, Fugard AJB, Deighton J, Gorzig A (2012) Editorial Commentary: routine outcome monitoring as part of children and young people’s improving access to psychological therapies (CYP IAPT)––improving care or unhelpful burden? Child Adol Ment Heal 17:129–130
Hatfield DR, Ogles BM (2004) The use of outcome measures by psychologists in clinical practice. Prof Psychol Res Pr 35:485–491
Trauer T, Gill L, Pedwell G, Slattery P (2006) Routine outcome measurement in public mental health––what do clinicians think? Aust Heal Rev A Publ Aust Hosp Assoc 30:144–147
Close-Goedjen JL, Saunders SM (2002) The effect of technical support on clinician attitudes toward an outcome assessment instrument. J Behav Heal Serv Res 29:99–108
Garland AF, Kruse M, Aarons GA (2003) Clinicians and outcome measurement: what’s the use? J Behav Heal Serv Res 30:393–405
Mathai J, Anderson P, Bourne A (2004) Comparing psychiatric diagnoses generated by the strengths and difficulties questionnaire with diagnoses made by clinicians. Aust New Zeal J Psychia 38:639–643
Garralda EM (2009) Accountability of specialist child and adolescent mental health services. Br J Psychia 194:389–391
Baruch G, Vrouva I, Fearon P (2009) A follow-up study of characteristics of young people that dropout and continue psychotherapy: service implications for a clinic in the community. Child Adolesc Ment Heal 14:69–75
McCarthy-Hoffbauer I, Leach C, McKenzie I (2006) Deliberate self harm in children and adolescents. Ment Heal Learn Disabil Res Pr 3:107–125
Lee WL, Jones L, Goodman R, Heyman I (2005) Broad outcome measures may underestimate effectiveness: an instrument comparison study. Child Adolesc Ment Heal 10:143–144
Walter G, Cleary M, Rev JM (1998) Attitudes of mental health personnel towards rating outcome. J Qual Clin Pr 18:109–115
Bickman L (2008) A measurement feedback system (MFS) is necessary to improve mental health outcomes. J Am Acad Child Adolesc Psychia 47:1114–1119
Martin AM, Fishman R, Baxter L, Ford T (2011) Practitioners’ attitudes towards the use of standardized diagnostic assessment in routine practice: a qualitative study in two child and adolescent mental health services. Clin Child Psychol Psychia 16:407–420
Harnett PH, Loxton NJ, Sadler T, Hides L, Baldwin A (2005) The health of the nation outcome scales for children and adolescents in an adolescent in-patient sample. Aust New Zeal J Psychia 39:129–135
Bickman L, Rosof-Williams J, Salzer MS et al (2000) What information do clinicians value for monitoring adolescent client progress and outcomes? Prof Psychol Res Pr 31:70–74
Willis A, Deane FP, Coombs T (2009) Improving clinicians’ attitudes toward providing feedback on routine outcome assessments. Int J Ment Heal Nurs 18:211–215
Moran P, Kelesidi K, Guglani S, Davidson S, Ford T (2012) What do parents and carers think about routine outcome measures and their use? A focus group of CAMHS attenders. Clin Child Psychol Psychia 17:65–79
This study was funded by National Institute for Health Research (NIHR) Collaborations for Leadership in Applied Health Research and Care––Nottinghamshire, Derbyshire, Lincolnshire (CLAHRC-NDL)
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
The paper is not a human or animal study so ethical approval was not required. However, all work was undertaken under the auspices of the CATO (Consensus Assessment and Treatment Outcome Measures) study, which has ethical approval.
About this article
Cite this article
Hall, C.L., Moldavsky, M., Taylor, J. et al. Implementation of routine outcome measurement in child and adolescent mental health services in the United Kingdom: a critical perspective. Eur Child Adolesc Psychiatry 23, 239–242 (2014). https://doi.org/10.1007/s00787-013-0454-2
- Child and adolescent psychiatry
- Routine outcome measurement