A Qualitative Exploration of Patient and Clinician Views on Patient Reported Outcome Measures in Child Mental Health and Diabetes Services
- 2k Downloads
Patient Reported Outcome Measures (PROMs) are increasingly being recommended for use in both mental and physical health services. The present study is a qualitative exploration of the views of young people, mothers, and clinicians on PROMs. Semi-structured interviews were conducted with a sample of n = 10 participants (6 young people, 4 clinicians) from mental health services and n = 14 participants (4 young people, 7 mothers, 3 clinicians) from a diabetes service. For different reasons, young people, mothers, and clinicians saw feedback from PROMs as having the potential to alter the scope of clinical discussions.
KeywordsPatient Reported Outcome Measures (PROMs) Mental health Diabetes Qualitative
The routine use of Patient Reported Outcome Measures (PROMs) is recommended by healthcare policy for a range of long-term conditions in the United Kingdom (UK), including services for young people with mental health problems or diabetes (Department of Health 2011, 2012; Royal College of Paediatrics and Child Health 2012). PROMs have been defined elsewhere in this special issue (Edbrooke-Childs et al. 2014). In child mental health services, PROMs are being implemented to measure change in mental health symptoms and functioning (e.g., Revised Children’s Anxiety and Depression Scales: Chorpita et al. 2005) and to measure progress to achieving therapeutic goals (i.e., Goal Based Outcome measure: Law 2011; Wolpert et al. 2012). In child diabetes services, PROMs are being implemented to measure change in general wellbeing (e.g., Pediatric Quality of Life Inventory (PedsQL) Varni et al. 1999; Royal College of Paediatrics and Child Health 2012).
Evidence suggests that the routine use of PROMs may benefit shared treatment decision making between patients and clinicians, service satisfaction, and treatment progress monitoring (Batty et al. 2013; Bickman et al. 2011; De Wit et al. 2006; Skevington and McCrate 2012; Wolpert 2013). On the one hand, research has focussed on the development of PROMs and ensuring they measure salient issues to patients (Skevington and McCrate 2012). On the other hand, there is little research exploring patient and clinician attitudes to PROMs (Batty et al. 2013).
As reported elsewhere in this special issue (de Jong 2014; Douglas et al. 2014; Edbrooke-Childs et al. 2014; Fleming et al. 2014; Mellor-Clark et al. 2014), there are a number of barriers to implementing PROMs, mirrored in qualitative studies with patients and clinicians. Support is indeed needed for services to overcome these barriers and routinely use PROMs. As demonstrated in an audit of a child mental health service, the routine administration of PROMs doubled after one year, with the support of an active learning collaboration (Hall et al. 2013).
Firstly, the content and format of measures. This includes their structured format, the focus on problems or deficits, and the inability to capture the complexity of the young person’s experience and context.
Secondly, the process of using measures, as these may be perceived to interfere with therapeutic engagement.
Finally, the outcome of using measures, considering whether data obtained from measures may be more relevant and useful to services than to patients.
A particular issue for the outcome of using measures is that patients may be unwilling to reveal some topics or may feel that PROMs do not capture their concerns and priorities. Furthermore, clinicians may not view problems identified using PROMs as warranting a change in practice or referral because they may be seen as either an inevitable side effect of treatment or, on the contrary, as not being caused by treatment or presenting problems (Greenhalgh 2009; Marshall et al. 2006).
Young people have suggested that the use of PROMs may be encouraged by making measures more convenient and flexible, promoting ‘quick’, ‘simple’, ‘well-explained’, and ‘optional’ measures (Badham 2011, p. 13). How PROMs are used rather than the measures themselves may be a key determinant of attitudes, and young people appear to support PROMs when used after a rapport has been developed with the clinician (Stasiak et al. 2013). Clinicians report concerns around resourcing and a lack of time for and ownership of implementation, some of which may be offset by computerised implementation to ensure timely feedback to clinicians (Batty et al. 2013; Martin et al. 2011; Murphy et al. 2011).
Given the increasing use of PROMs in services for young people (Department of Health 2011, 2012; Royal College of Paediatrics and Child Health 2012), research is required to understand young people’s, parents’, and clinicians’ views on PROMs. In particular, understanding perspectives on the barriers and facilitators to implementing PROMs, and how these may be similar and different across mental and physical healthcare settings, may help inform how to appropriately tailor implementation.
Aims of the Present Research
To the best of our knowledge, no study has explored the similarities and differences in perspectives of patients and clinicians in settings related to long-term mental and physical conditions in young people. To this end, the aim of the present research was to explore the views of young people, parents, and clinicians from child mental health and diabetes services to understand their views on PROMs, the implementation of PROMs, and the barriers to implementation.
Mother’s ethnicity (n)
Years accessing services (n)
Young people from child mental health services (6)
Face-to-face in focus group (6)
Male (5), female (1)
White British (4)
Mixed Ethnic Group (2)
Young person-mother dyads from diabetes services (4)
Male (2), female (2)
White British (3)
Mixed Ethnic Group (1)
White British (3)
White Other (1)
Mothers from diabetes services; child not interviewed (3)
Individually face-to-face (2) or by phone (1)
Male (1), female (2)
White British (2)
Mixed Ethnic Group (1)
White British (2)
White Other (1)
Young people and mothers were recruited using advertisements with voluntary sector organisations. Clinicians were recruited using the research team’s networks of clinicians leading work to introduce PROMs. Interviews took place over 6 months during 2011 and were conducted by a senior qualitative researcher (KCT), either by telephone or face-to-face (see Table 1 for details). Interviews were conducted individually, in child-mother dyads, or in a focus group, which was held with members of a child mental health service user participation group.
In all interviews, participants were asked about the information clinicians should seek in order to understand patients’ priorities for care and about their views on the implementation of PROMs. Discussions were recorded and transcribed, then, using the constant comparative method, data were coded on themes from previous studies and those arising directly from the accounts of participants in this study. The data was summarised in charts by case and theme to facilitate identification of patterns (and disconfirming cases) within and across cases (Ritchie et al. 2003; Glaser and Strauss 1967).
Young people and their mothers had not routinely used PROMs in their treatment, as the study was conducted before PROMs had been implemented in these settings. Hence, in child mental health services, participants were asked for their views on a symptom checklist currently used in these services (Jones et al. 2013); in the diabetes service, participants were asked for their views on the PedsQL (Varni et al. 1999) currently used in these services.
The study was approved by the School of Health Sciences Research Ethics Committee at City University, London. The study was conducted in line with the British Psychological Society Code of Human Research Ethics (BPS 2010) and all participants gave written informed consent prior to participation. On meeting the participants, the researcher emphasised that participation was voluntary and that they could stop taking part at any point without giving a reason; no one withdrew. Young people and mothers were given a £10 voucher as a thank you for participation.
All interviews were audio-recorded and transcribed verbatim. Data were then summarised and explored using thematic analysis, by comparing patterns, similarities, and differences across cases. Data analysis was performed by the second author (KCT). Primary and secondary themes are explored, below.
Scope of Clinical Discussions
Clinicians across both settings saw PROMs as a tool for altering the scope of clinical discussions, in turn creating opportunities to tailor care more closely to individual need. Most envisaged implementing PROMs alongside measures of patient experience and, as recommended (Greenhalgh 2009), individualised goal setting. Some clinicians felt that asking patients for their opinion might be partially empowering.
In a context where problems and ‘possible ways forward’ may be less than clear cut (Clinician 1), mental health clinicians hoped that PROMs might enable interventions to be more tightly focused on the needs of the family, promoting ‘better, quicker outcomes’. Here, there was a drive to hear families’ perspectives and priorities in the first appointment, where they might be offered ‘two or three choices’ for intervention (Clinician 1) and progress could then be monitored using PROMs at every session. However, young people saw this focussing of treatment as potentially threatening to the patient–clinician relationship (see ‘pace of clinical discussions’ below).
Young people and mothers in the diabetes service worried that widening the scope of clinical discussions, through PROMs, would attract unwarranted professional attention to emotional issues, potentially opening families up to stigma associated with mental health problems. Mothers also feared professional scrutiny of the quality of care they were providing for their children, if emotional issues were raised.
Young people with diabetes reported that describing their experiences in the clinic could be a challenge: ‘I have to think about [what I say] for a very long time and then I can’t be accurate with my words’ (Young Person 1, 14, diabetes). There were mixed feelings as to whether this would be supported by the use of PROMs. For instance, Young Person 2 (9, diabetes), reported some confidence that the PROM might help her ‘show what I feel like… so [clinicians] can help me,’ while Young person 1 disagreed: ‘I put it down because it… matched how I felt but I’m not sure how to approach it now I have written it.’
Clinicians and service users echoed concerns voiced in previous studies (Moran et al. 2012; Norman et al. 2013; Stasiak et al. 2013) about the potential for PROMs to impact negatively on the patient-clinician relationship.
Safe to Reveal to Clinicians
Young people and mothers in both settings expressed uncertainty about what PROMS would be used for, and on the basis of this, what would be appropriate, or safe, to reveal to clinicians (Stasiak et al. 2013). Young people were concerned about what their answers might reflect on their clinician and his or her practice: ‘I wouldn’t know if it would be rude to tick like ‘worse’ or ‘much worse’ (Young Person 3, 14, CAMHs). Young people in mental health services were concerned that PROMs might be used to narrow access to care; e.g., ‘If I tick ‘much worse’ then I don’t know if [that means] “I don’t need this service”, I don’t know, I’m very confused,’ (Young Person 4, 14, CAMHS). What is safe to reveal with PROMs may reflect a general fear of the unknown that children and their parents report when first attending CAMHS (Bone et al. 2014).
Similarly, clinicians from both settings felt that it would be important to provide patients with a good rationale for PROMs, explaining how information would be used, and to provide feedback from information obtained from PROMs.
Pace of Clinical Discussions
If he just suddenly asks you, ‘So tell me about your problem,’ you wouldn’t just say, like, ‘Okay’. You’d tell your best friend or someone you are really, really close to but you wouldn’t tell this random person you’d just met’ (Young Person 3, 14, CAMHs).
In contrast, mothers from the diabetes service were worried that changing the scope of clinical discussions to encompass emotional problems raised by PROMS might detract from time available to discuss issues relevant to diabetes management or the ‘objective hardcore,’ (Mother 2).
Tension Between the Fixed and the Fluid
The thing is, I think that the reason I wouldn’t want to do it beforehand, is because I might start doubting whether my answers were right…if I do it quickly, on the paper, I’ve done it and I can’t go back and change it all. (Young Person 5, 10, Diabetes).
Clinicians suggested that PROMs should be made more usable, dovetailing with previous studies (Norman et al. 2013); e.g., ‘something relatively uninvasive, something fairly quick…something much more easily useable for the everyday clinician,’ (Clinician 2). Clinicians also highlighted the need for extensive information technology and administrative support for implementation, especially to ensure timely feedback of results to clinicians and young people: ‘what happens in a week could very much change your life quite quickly as a teenager’ (Clinician 7).
Across settings, young people and mothers shared a view that measures could be improved by using a more unstructured, flexible format and by making them more visually appealing with electronic administration. Some suggested using pictures and symbols instead of numbers and words. Young Person 6 remembered a CAMHS worker using a chart where ‘there was loads of different people sat up in a tree and loads of different faces and you… circle the one you’re feeling’. Young Person 2’s mother (diabetes) suggested: ‘it would be quite interesting to almost do it pictorially…you move your needle-pricker along a sliding scale.’
[Benefits of] the outcome monitoring [for service development]… become very slightly further away from the client, and then get back to them. So if you find that we’re not particularly good at eating disorders, then it may have an impact on sort of training in the service, that we get better at doing eating disorders, and then eventually a cohort of clients that come through get a better service (Clinician 1).
The aim of the present research was to explore the views of young people, parents, and clinicians from child mental health and diabetes services to understand their views on PROMs, the implementation of PROMs, and the barriers to implementation.
For different reasons, young people, mothers, and clinicians viewed PROMs as having the potential to alter the scope of clinical discussions. On one hand, clinicians reported this as being an opportunity to better tailor care to individual need and to do so from the very start of treatment. On the other hand, young people in mental health services reported concerns that this would alter the pace of clinical discussions, prioritising a focus on treatment at the expense of therapeutic relationship building. Young people and mothers in the diabetes service were concerned about clinical discussions moving away from physical to mental health concerns, bringing with it increased stigma and scrutiny. Young people and mothers from both settings raised concerns about how data obtained from PROMs would be used and what would be safe to reveal using measures. In particular, young people from mental health services were concerned that responses would be used to narrow access to services.
Other potential barriers were information technology and administrative support and the tension between the fixed structure of PROMs and the need to capture the fluidity of patients’ experiences. Still, some clinicians valued the capacity of structured instruments to draw comparisons of outcomes across patient groups, which it was hoped would facilitate service development.
Limitations should be considered when interpreting the findings of the present research. First, the sample was small, and conclusions may not generalise beyond the present sample. Second, participants were recruited using the research team’s professional networks and advertisements, which risk self-selection bias. Finally, interviews with young people with diabetes and their mothers were conduct in dyads, potentially limiting young people’s freedom of expression.
Notwithstanding the above limitations, the present research is the first to explore and triangulate the views of young people, mothers, and clinicians from child mental health and diabetes services to understand their views on PROMs, the implementation of PROMs, and barriers to implementation.
Findings from the present research highlight the need for well-planned and resourced support for PROM implementation, including guidelines to help clinicians know how to administer and interpret PROMs (Devlin and Appleby 2010; Greenhalgh 2009; Greenhalgh et al. 2005; Law 2012). As mentioned by clinicians from both settings, young people and their parents need explanations of the rationale for using PROMs, as they may have a range of concerns about the content, process, and outcome of implementing measures. If clinicians are to use PROMs to guide practice with their patients, they may need to find ways to foster young people’s and parents’ ownership of measures and data. Using PROMs within a shared decision making context may help achieve this (Law 2012; Wolpert 2013).
To do so, clinicians should discuss with young people and parents when to use- and not use-measures. If it is decided to use measures, discussions should explore which areas to focus on in sessions and correspondingly, what measures to use. The potential clinical utility of measures should be considered in terms of monitoring and reviewing treatment progress. Such data obtained from PROMs can be used to look for off track cues so clinicians can discuss with young people and their parents instances when treatment might not be progressing as expected. Action plans can then be devised to try new methods to redress treatment progress, with PROMs used to continue to monitor the impact of changes.
Findings of the present research suggest that young people, mothers, and clinicians might all view PROMs as having the potential to alter the scope of clinical discussions. These alterations need to be carefully discussed with young people and mothers as they may view potential alterations differently to clinicians. Clinicians in the present study viewed PROMs as potentially useful for tailoring care to the needs of the patient from the outset of treatment. However, young people in mental health services were concerned that this may take time away from therapeutic relationship building in the initial stages of treatment. Conversely, young people and their mothers in the diabetes service were concerned that emotional issues raised by PROMs may detract from physical healthcare.
For different reasons, young people, mothers, and clinicians in both mental and physical healthcare viewed PROMs as having the potential to alter the scope of clinical discussions. Before implementing PROMs, clinicians need to be aware of the associated strengths and barriers this may bring, so they can carefully communicate these to the young people and parents in their care.
The authors would like to thank the young people, mothers and professionals who took part in the study, and those who helped recruit participants; also members of the Policy Research Unit in the Health of Children, Young People and Families: Terence Stephenson, Catherine Law, Amanda Edwards, Ruth Gilbert, Steve Morris, Russell Viner and Catherine Shaw. The study was funded by the Department of Health Policy Research Programme.
The study was funded by the Department of Health Policy Research Programme. This is an independent report commissioned and funded by the Department of Health. The views expressed are not necessarily those of the Department. The findings of this study have been reviewed by the Department of Health Policy Research Unit in the Health of Children, Young People and Families.
Conflict of interest
Miranda Wolpert is paid Director (1.5 days a week) of CORC: a not-for-profit learning collaboration of child mental health providers committed to developing models of outcome evaluation in child and adolescent mental health services whose members use a range of PROMs and PREMs. Miranda is also currently seconded 1 day a week to the Department of Health as National Informatics Advisor for CYP IAPT in which capacity she has been involved in developing and implementing models of the use of PROMs and PREMS across CAMHS. Authors have no other interests relevant to the submitted work other than the fact that as researchers working on health care and user involvement, publications in this area potentially strengthen the field, and grant applications.
- Badham, B. (2011). Talking about talking therapies: thinking and planning about how to make good and accessible talking therapies available to children and young people. Retrieved August 26, 2014 from http://www.iapt.nhs.uk/silo/files/talking-about-talking-therapies.pdf.
- Batty, M. J., Moldavsky, M., Foroushani, P. S., Pass, S., Marriott, M., Sayal, K., et al. (2013). Implementing routine outcome measures in child and adolescent mental health services: From present to future practice. Child and Adolescent Mental Health, 18(2), 82–87. doi: 10.1111/j.1475-3588.2012.00658.x.CrossRefGoogle Scholar
- Bickman, L., Kelley, S. D., Breda, C., de Andrade, A. R., & Riemer, M. (2011). Effects of routine feedback to clinicians on mental health outcomes of youths: Results of a randomized trial. Psychiatric Services (Washington, D. C.), 62(12), 1423–1429. doi: 10.1176/appi.ps.002052011.CrossRefGoogle Scholar
- BPS. (2010). Code of Human Resarch Ethics. Leicester: The British Psychological Society.Google Scholar
- de Jong, K. (2014). Deriving effective implementation strategies for outcome monitoring from feedback theory. Administration and Policy in Mental Health, Current issue. Google Scholar
- De Wit, M., Delemarre-van de Waal, H. A., Alle Bokma, J., Haasnoot, K., Houdijk, M. C., Gemke, R. J. B. J., et al. (2006). Monitoring and discussing health related quality of life in adolescents with type 1 diabetes improves psychosocial well-being: A randomised controlled trial. Diabetes Care, 31, 8.Google Scholar
- Department of Health. (2011). Talking therapies: A four-year plan of action. London: Department of Health.Google Scholar
- Department of Health. (2012). IAPT three-year report: The first million patients. London.Google Scholar
- Devlin, N., & Appleby, J. (2010). Getting the most out of PROMs: Putting health outcomes as the heart of NHS decision-making. London: The Kings Fund and Office of Health Economics.Google Scholar
- Douglas, S., Button, S., & Casey, S. E. (2014). Implementing for sustainability: Promoting use of a Measurement Feedback System for innovation and quality improvement. Administration and Policy in Mental Health, Current issue. Google Scholar
- Edbrooke-Childs, J., Bickman, L., & Bickman, C. (2014). Introduction to the special issue on implementing routine outcome monitoring. Administration and Policy in Mental Health, Current issue. Google Scholar
- Edbrooke-Childs, J., Wolpert, M., & Deighton, J. (2014). Using Patient Reported Outcome Measures to Improve Service Effectiveness (UPROMISE): A service evaluation of training for clinicians to use outcome measures in child mental health. Administration and Policy in Mental Health, current issue. Google Scholar
- Fleming, I., Jones, M., & Bradley, J. (2014). Learning from a learning collaboration: The CORC approach to combining research, evaluation and practice in child mental health. Administration and Policy in Mental Health and Mental Health Services Research, Current issue. Google Scholar
- Glaser, B., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. New York: Aldine de Gruyter.Google Scholar
- Hall, C. L., Moldavsky, M., Baldwin, L., Marriott, M., Newell, K., Taylor, J., et al. (2013). The use of routine outcome measures in two child and adolescent mental health services: A completed audit cycle. BMC Psychiatry, 13, 270. doi: 10.1186/1471-244X-13-2701471-244X-13-270.CrossRefPubMedPubMedCentralGoogle Scholar
- Jones, M., Hopkins, K., Kyrke-Smith, R., Davies, R., Vostanis, P., & Wolpert, M. (2013). Current view tool: Completion Guide. London: CAMHS Press at EBPU.Google Scholar
- Law, D. (2011). Goals and Goal Based Outcomes (GBOs): Some useful information: CAMHS Outcome Research Consortium and Hertfordshire Partnership NHS Foundation Trust.Google Scholar
- Law, D. (2012). A Practical Guide to Using Service User Feedback & Outcome Tools to Inform Clinical Practice in Child & Adolescent Mental Health Some initial guidance from the Children and Young Peoples’ Improving Access to Psychological Therapies Outcomes-Oriented Practice (CO-OP) Group. Retrieved August 26, 2014 from http://www.iapt.nhs.uk/silo/files/a-practical-guide-to-using-service-user-feedback–outcome-tools-.pdf.
- Martin, A. M., Fishman, R., Baxter, L., & Ford, T. (2011). Practitioners’ attitudes towards the use of standardized diagnostic assessment in routine practice: A qualitative study in two child and adolescent mental health services. Clinical Child Psychology and Psychiatry, 16(3), 407–420.CrossRefPubMedGoogle Scholar
- Mellor-Clark, J., Cross, S., Macdonald, J., & Skjulsvik, T. (2014). Leading horses to water: Lessons from a decade of helping psychological therapy service use routine outcome measurement to improve practice. Administration and Policy in Mental Health, Current issue. Google Scholar
- Moran, P., Kelesidi, K., Guglani, S., Davidson, S., & Ford, T. (2012). What do parents and carers think about routine outcome measures and their use? A focus group study of CAMHS attenders. Clinical Child Psychology and Psychiatry, 17(1), 65–79. doi: 10.1177/13591045103918591359104510391859.CrossRefPubMedGoogle Scholar
- Norman, S., Dean, S., Hansford, L., & Ford, T. (2013). Clinical practitioner’s attitudes towards the use of Routine Outcome Monitoring within Child and Adolescent Mental Health Services: A qualitative study of two Child and Adolescent Mental Health Services. Clinical Child Psychology and Psychiatry. doi: 10.1177/1359104513492348.PubMedGoogle Scholar
- Ritchie, J., Spencer, L., & O’Connor, W. (2003). Carrying out qualitative analysis. Qualitative research practice: A guide for social science students and researchers. London: Sage.Google Scholar
- Royal College of Paediatrics and Child Health. (2012). National Paediatric Diabetes Audit call for applications for members and chairs of working groups. London: Royal College of Paediatrics and Child Health.Google Scholar
- Stasiak, K., Parkin, A., Seymour, F., Lambie, I., Crengle, S., Pasene-Mizziebo, E., et al. (2013). Measuring outcome in child and adolescent mental health services: consumers’ views of measures. Clinical Child Psychology and Psychiatry, 18(4), 519–535. doi: 10.1177/13591045124608601359104512460860.CrossRefPubMedGoogle Scholar
- Wolpert, M., Ford, T., Trustam, E., Law, D., Deighton, J., Flannery, H., et al. (2012). Patient-reported outcomes in child and adolescent mental health services (CAMHS): Use of idiographic and standardized measures. Journal of Mental Health, 21(2), 165–173. doi: 10.3109/09638237.2012.664304.CrossRefPubMedGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.