Round 1: Item Generation
Purposive sampling was used to recruit young carers aged 11–18, mainly from young carers’ centres in North West England. Additionally, several participants were recruited from an organisation that works with young people who have a sibling with a life-limiting health condition, and a local university Service Users and Carers group. As levels of comprehension can vary substantially at different ages (Gibson, 2007; Stewart & Shamdasani, 2014), separate information packs for two age groups, 11–15, and 16–18, were developed following consultation with several young people. Potential participants were identified by staff at the various groups and given the relevant information and recruitment packs about the study to take home and consider. All of the 26 recruited participants provided written assent or consent to take part, with those aged under 16 also providing written consent from their parent or guardian. The average age of the participants was 14.7 years, with 15 of the participants reporting their gender as female and 11 as male. In total, 12 participants lived in a household with two parents or legal guardians and 14 lived in a lone parent household. Fifteen of the care recipients were either one or more of the young carer’s parents, eight care recipients were a sibling, and three were a combination of both.
Seventeen individual interviews were conducted at either a young carers’ centre or support group, or for those who were not members of a young carers’ centre, at their family home. Data collected from those interviews provided a balance to data collected via participants at dedicated young carers’ groups, as different perspectives and experiences were given, thereby reducing any potential for bias from purposive sampling at those sites (Bryman, 2016; Polit & Beck, 2017). Additionally, a focus group comprising of nine participants aged 12–14 was held at one of the venues. A semi-structured guide was used for data collection (see Table 1) and data were digitally recorded and transcribed verbatim. Information about support services was provided to all participants.
Table 1 Round 1 topics and questions in the semi-structured guide Round 1 Data Analysis and Survey Development
A thematic analysis approach was used to develop themes identified in the interview and focus group data (Braun & Clarke, 2006). Two main overarching themes were identified: the young carers’ situation and caring role within the family home; and their own health and wellbeing. Key factors in the themes were discussed, reviewed and refined by all members of the research team. Further development of the themes identified five sub-themes, or domains, as identified in Fig. 2. In order to reduce potential bias from collecting data from one regional area in the UK, existing literature was revisited to ensure the items developed for the survey were comprehensive. Altogether, 37 items (questions) were developed from the five domains for R2 of the Delphi survey. The purpose of the next rounds of the survey was for young carers aged 11–18 and relevant health, social care and education-based professionals, to rate the importance of the items and begin the process of reaching consensus on the most important items to be included in the CAT-YC. Respondents were asked to rate the items on a Likert scale from (1) ‘not at all important’ to (5) ‘extremely important.’ A comment box was provided next to each domain to allow respondents to express any issues, or to highlight any items they felt were missing from the CAT-YC. A separate demographics section was added to the survey to capture relevant information about the respondents. Additionally, a final section was added, asking respondents whether they felt it was important for young carers’ needs to be assessed, and if so, how often they should be reviewed. Respondents for the next round of the survey were also asked to choose from a multiple-choice list of who they felt should carry out screenings and assessments. Finally, respondents were asked to rank the five domains according to their perception of least to most important (Knighting et al., 2015, 2016). The survey was created online using SurveyMonkey® and was also available in paper format. Prior to launching the survey, a small pilot study to check readability and usability was undertaken with professionals and young carers (n = 6) at organisations who had taken part in R1, as recommended for Delphi studies by Novakowski and Wellar (2008).
Round 2
A purposive sampling approach was used to recruit young carers aged 11–18 and professionals working with young carers. Emails with information about the study and an invitation to participate were sent out to 142 national young carers’ organisations and charities known to support young carers for inclusion in newsletters. Staff at those organisations were asked to cascade information to other relevant professionals and young carers aged 16–18, as appropriate (Hunt & Lathlean, 2015). Furthermore, the social media site Twitter was used to advertise the study. However, social media was not used to recruit young carers aged 11–15, due to guidelines recommending that young people under 14 should not have social media accounts (Schurgin & Clarke-Pearson, 2011; UK Council for Child Internet Safety, 2016). Therefore, a purposive sampling approach was used to recruit young carers aged 11–15 to take part in small consensus group meetings, via young carers’ support groups and a national charity supporting young carers (Green & Thorogood, 2014; Bryman, 2016). Using such an approach, in tandem with the Delphi process, ensured that all invited young carers who wished to participate had the opportunity to complete the survey and provide information and feedback in a supportive environment (Bailey et al. 2015).
The R2 survey was completed in July 2018 by 68 professionals and 28 young carers (n = 96). The professionals consisted of a range of staff working either directly or indirectly with young carers, in the paid and voluntary sectors. The average age of the young carers was 14.4 and more than half of them reported providing care for 21 or more hours per week. The majority of respondents (18) primarily cared for their mother, five cared for their father, three cared for a sibling and two cared for a grandparent. The main reasons for their family member needing care were due to a mental health condition (57%), a physical health condition (39%) or a physical disability (32%), although some respondents selected more than one reason. The same methods of analysis were used for both the online Delphi surveys and consensus group surveys. Qualitative data were analysed thematically, as in R1 (Braun & Clarke, 2006). Quantitative data were entered into the IBM SPSS Statistics, Version 25, for analysis (Armonk, NY; IBM Corp, 2017). Descriptive statistics, relating to profession (for professionals), or age, gender and length of time as a carer (for young carers), were used to describe the samples according to the demographic information gained from the surveys (Field, 2013; Bryman, 2016). Data concerning the collective judgements of the respondents were analysed using measures of central tendency, frequency and levels of dispersion (Field, 2013). The following pre-determined consensus criteria were used:
Criterion to Accept an Item
At least 60% of the young carer and professional cohorts rated an item as (4) (very important) or (5) (extremely important). Accepted items were removed from the survey for the next round as they did not require any further rating.
Criterion for Re-Rating an Item
If 60% of one cohort rated an item as (4) (very important) or (5) (extremely important) but the other cohort did not, suggesting disagreement between cohorts, they were retained for re-rating in R3.
Criterion for Rejecting an Item
Any items that did not meet the 60% criterion in the total sample were removed from the survey.
Altogether, 13 items met the criterion for acceptance, one item was rejected, and 23 items were retained for re-rating in R3.
Round 3
The same sampling strategy and data collection method used in the previous round was used for R3. Additionally, controlled feedback was given for the remaining 23 items using the group median score to show respondents the level and range each item was rated at previously (Keeney et al., 2011). Items that had reached consensus in R2 did not form part of the survey for R3 but were displayed under each domain as a reminder of what had been previously accepted (Keeney et al., 2011).
The R3 survey was completed in October 2018 by 74 professionals and 36 young carers (n = 110) and data were analysed quantitatively and qualitatively as in R2. Demographic results were similar to those in R2, as many of the respondents were likely to have been the same individuals. The same criterion for acceptance was used, resulting in an additional 13 items to be added to the previously accepted items from R2. As in R2, one item met the rejection criteria and was therefore immediately rejected. There was still disagreement between the cohorts for nine items. Therefore, as consensus had not been achieved after both rounds, these items were also rejected, in accordance with the pre-determined consensus criteria (McKenna et al., 2015). The mean and standard deviation (SD) were calculated for each of the 26 items that reached consensus in R2 and R3 in order from ‘most important’ to ‘least important’ (McKenna et al., 2015). The items were then sent for a fourth round to an expert panel in January 2019, to comment on the list of top ranked items from the Delphi survey findings and choose and rank a final list of ten items. The rationale for having ten items was based on the pragmatic need to not overburden young carers or professionals working with them by having too many questions in the CAT-YC.
Round 4
The expert panel comprised of eight professionals from national or regional organisations that have a key strategic role in young carer support, and eleven young carers aged 16–18 who were identified and approached by gatekeepers at organisations that took part in earlier phases of the study. The rationale for only including young carers aged 16–18 was based on the panel being conducted virtually, and as reported earlier, young carers aged 11–15 were not approached or encouraged to take part in the study online (UK Council for Child Internet Safety, 2016). The items were listed in a three-section survey created through SurveyMonkey®, with separate demographic sections for each cohort. The first section asked participants to review the list of items and domains and provide feedback. The second section asked participants to choose and rank their top ten items; the final section asked who should use the CAT-YC and how often, and for demographic details to be provided.
Overall, there was a high level of consensus between both the professional and young carer cohorts, with each cohort ranking the same eight items in their top ten. It is interesting to note that three of the items were from Domain 1 (Understanding the Caring Situation), three from Domain 2 (Details of the Care Provided by the Young Person), three from Domain 3 (The Young Carers Own Health and Social Wellbeing), none from Domain 4 (The Young Carers Identity and Self-Esteem) and one from Domain 5 (Education and Future Plans).
Feedback was very positive about the comprehensiveness of the items:
Very comprehensive [...]appears holistic and should be suitable for all professionals to use (Professional-06)
I felt like all the items should definitely be included (Young Carer-04)
Participants in the expert panel were asked their views of how often the needs of young carers should be reviewed, as a minimum. ‘Every six months’ was the option most frequently selected by participants from both cohorts, with almost half of the participants overall choosing this option. Table 2 shows the results of who participants felt should be responsible for using the CAT-YC with young carers.
Table 2 Views on which professionals should use the CAT-YC (Expert Panel) Creation of the CAT-YC
Prior to consulting with professionals and young carers, an initial draft version of the CAT-YC was created, based on similar design principles to those of the original Carers’ Alert Thermometer (CAT) for adult carers (Knighting et al., 2015, 2016). The first section of the draft CAT-YC was developed to include space for the young carer’s demographic information, education details, parent or guardian’s contact details, family composition and basic pertinent information about the person they care for. The second section contained an initial identifying question to ascertain if the young person was providing care for a family member and if so, whether they shared care with another person. This was followed by a question asking them to rate, using a traffic light system of green (low), amber (medium) or red (high) emojis, how much help or support they needed with each of the top ten items from the findings in the Delphi survey, as shown in Table 3. Section three contained an alert thermometer to record the number of high or medium alerts, thereby providing a visual overview of the areas needing an urgent response or monitoring. Section four had a ‘suggested next steps’ section for staff, with space to include appropriate guidance for addressing each item scoring a high or medium alert. Section five had space to record up to three priority alerts requiring action, together with boxes for any immediate action taken, next steps, name of the person responsible for following up the action and a date for when the action would be followed up. The final section had space for when and who would conduct the next review, information regarding consent for use of data and space for the young carer’s signature. Similarly, a user guide for staff was developed to explain how to administer and complete each section of the tool. Information was also included in the guide to remind practitioners to refer to any appropriate safeguarding policies and to discuss storage and sharing of the CAT-YC, according to their own organisational policies and procedures.
Table 3 Top ten items for inclusion in the CAT-YC Consultation and CAT-YC Pilot Evaluation
Gatekeepers at organisations who had taken part in previous phases of the study were approached in March 2019 and invited to take part in a brief validity pilot to test the readability and usability of the CAT-YC. First, four members of staff and two young carers (n = 6) took part in a consultation exercise. Following this, slight changes and refinements were made to the layout of the CAT-YC, which is on one double-sided sheet of A4. The CAT-YC was then piloted in April 2019 by three organisations working with young carers in North West England. Semi-structured interviews were conducted with staff (n = 6) and young carers (n = 4) to explore their experiences and views of using it. Staff who took part had a mean average of eight years of experience working with young carers; young carers were aged 15–17 and had a mean average of four years of experience in their caring role. Data from the interviews were transcribed and analysed using content analysis (Hsieh & Shannon, 2005; Elo & Kyngäs, 2008).
An evaluation of the findings indicated the CAT-YC was comprehensive, enabled discussion about issues of concern to the young carer, and would be a valuable tool to identify current as well as future areas of concern for the young carer. Participants liked the clear instructions and visual appearance of the CAT-YC. In particular, one staff member commented on how the young carers appreciated the use of red, amber and green emojis to indicate high, medium and low levels of need in section two:
It was easy to fill in and the young people liked the faces, rather than having the tick box in the forms we currently use – it’s good that it measures the level, rather than just yes/no (Staff 4)
Similarly, the practical focus of the CAT-YC was unanimously liked, particularly with regards to recording action points and accountability:
The action plan was nice to finish off with...it felt like something was happening, not just talking about it (Young Carer 2)
Particularly liked Section 5, especially for older young carers...who is responsible and the date, so the young person has it in their head what’s going to happen (Staff 1)
During the pilot study period, the CAT-YC successfully identified a number of alerts. Consequent actions included practical solutions, e.g. contacting school about a bullying situation, helping a young carer create a CV, and making an action plan for emergencies for a young carer whose parent has epilepsy. Other actions included signposting for additional support or services that could be helpful. Interestingly, one staff member reported that completing the CAT-YC with a young carer made them realise that some of the issues raised could be addressed immediately, without much effort. Typical completion time for the first use of the CAT-YC was approximately 20 min, although participants reported that this became quicker with increased use.
Participants recommended adding a reminder after the screening questions for staff to check whether there was anything else the young person wanted to talk about, or record for themselves on the CAT-YC. Space was therefore provided after the planning section for both staff and young carers to make notes or record pertinent information, if required. The CAT-YC is free to use by non-profit organisations and will be available for dissemination and use with supporting guidance from the project website, following a larger validity pilot currently taking place.