FormalPara Key Points for Decision Makers

Children with life-limiting conditions find brief measures with a short recall and visually appealing response format most relevant, acceptable and feasible for use.

Patient-reported outcome measures for children with life-limiting conditions should be available in paper and electronic formats.

To enhance acceptability of patient-centred outcome measure use in children with life-limiting conditions, they should be administered in conjunction with a face-to-face interaction with a health or social care professional.

1 Introduction

A patient-reported outcome measure (PROM) is defined as any measure of a patient’s health status, elicited directly from the patient without interpretation of the patient's response by a clinician or anyone else [1]. PROMs are standardised, validated questionnaires that are completed by patients to ascertain perceptions of their health status, perceived level of impairment, disability and well-being [2, 3]. Many palliative care patients, including children with life-limiting conditions, are too unwell or cognitively unable to self-report on their own health outcomes [4]. A measure that allows for proxy completion is required. Together PROMs and proxy-reported measures are termed patient-centered outcome measures (PCOMs) [5]. Within adult palliative care, PCOMs have been shown to improve service quality, increase referrals and lead to better symptom recognition and quality of life [4]. World-wide there are approximately 21 million children and young people (hereafter ‘children’) with life-limiting and life-threatening conditions (hereafter ‘life-limiting’) who could benefit from palliative care [5]. There are currently no suitable PCOMs to measure palliative care symptoms and concerns in this population outside of sub-Saharan Africa [6, 7]. Development of a validated measure for children is needed to realise the benefits of PCOM use that have been demonstrated with adults [8].

Guidance on methodological standards and quality criteria for PROM development have been published by the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) and Rothrock [9,10,11]. The COSMIN standards on evaluating the content validity of PROMs consider three aspects: relevance, comprehensiveness and comprehensibility [9]. During PROM development, it is also important to ascertain acceptability and feasibility within the population it is intended for [9, 11, 12]. Attention to the preferences and ability of the target population with regard to recall period, response options/format, mode of administration, length and ease of completion and administration increases the likelihood of use and implementation in routine practice [3].

Children with life-limiting conditions are often excluded from research participation due to the presumption that it will result in undue burden [13,14,15]. This presumption is not supported by empirical data [16] and has resulted in very little primary evidence on symptoms, concerns and care priorities in this population [14]. Past healthcare experience may impact upon opinions of intrinsic features of outcome measures; thus, it is important to involve children with life-limiting conditions in developing a PCOM. However, much of the existing data reflects the proxy opinions of parents, carers and healthcare professionals [17].

A recent systematic review showed that evidence regarding recall period, response scale format and administration mode is largely confined to either healthy children or those with chronic or oncological conditions with a good prognosis [18]. This (albeit limited) evidence suggests that children prefer visually appealing measures, require a short recall period of a few days to a week and should be offered the option of electronic measures [18]. Children with health conditions have different conceptions of health and illness compared to their healthy peers, due to greater exposure to medical care [19], and the nature and therapeutic interventions of different diseases [20]. They may also need different considerations in order to practically and conceptually engage in measure completion. Therefore, it is important to design measures that are suitable for their use and can capture their experience.

This study is part of a programme of work to develop the Children’s Palliative care Outcome Scale (C-POS), a child-centred outcome measure for use in paediatric palliative care within the UK. Previous sequential outputs include two systematic reviews (establishing the need for a new PCOM [8], identifying response formats and administration modes used in PCOMs for children [10, 21]) and primary qualitative data identifying symptoms, concerns and care priorities (the sample included children and young people, health and social care professionals, siblings, parents and commissioners) [22]. The aim of this analysis of the primary data is to identify preferences for PCOM design (recall period, response format, length, administration mode) to improve feasibility, acceptability, comprehensibility and relevance of a child and family-centred outcome measure, among children with life-limiting conditions and their family members. The results will be used to inform C-POS measure development, prior to cognitive testing and psychometric validation.

2 Methods

2.1 Study Design

A cross-sectional, semi-structured, qualitative interview study was conducted. This is reported in accordance with the Consolidated Criteria for Reporting Qualitative Studies (COREQ, see supplementary file 1) [23]. This study was conducted from a critical realist perspective, which allows the researcher to move beyond preferences shared by multiple participants, towards understanding the reasons for these preferences [24].

2.2 Setting

Participants were recruited from six hospitals and three children’s hospices in England and Northern Ireland.

2.3 Sampling and Recruitment

Inclusion criteria were as follows: children and young people (5–17 years old) with any life-limiting condition [25]; parents/carers responsible for the primary care needs of a child < 18 years old with a life-limiting condition; siblings (5–17 years old) of children and young people with a life-limiting condition. Siblings were included in order to gain the perspectives of healthy children and those who live with children with life-limiting conditions. Participants did not have to be recruited as family units.

2.4 Exclusion Criteria

The exclusion criteria for children and young people were as follows: unable to communicate via an in-depth interview, use of ‘draw and talk’ or play methods or via their parents; spoke a language not supported by NHS translation services; currently enrolled in any other study; unable to give consent/assent.

The exclusion criteria for parents/carers and siblings were as follows: unable to give consent/assent; spoke a language not supported by NHS translation services.

Purposive sampling was used to ensure maximum variation in the key characteristics of age and diagnosis.

2.5 Data Collection

Semi-structured interviews were conducted using a topic guide (supplementary file 2) that began with rapport-building questions, followed by questions about what mattered most (symptoms, concerns and care priorities) to individuals and their family, in order to inform content validity of the C-POS. Play and drawing were offered to children to aid interviews [26]. Following this, participants were asked how we could best measure the things that mattered in terms of response format, recall period and measure administration. Participants were given examples of response formats to help frame their answers and explore their interpretation and preference of these. These included a 0- to 10-point numerical rating scale, the Wong-Baker faces scale (a series of six faces ranging from a happy face at 0 to a crying face at 10) [27], Likert scales anchored by numbers and faces [28, 29] and the pain block scale (concrete ordinal picture-based scale, shaped as toy blocks) [30]. Participants were also asked to suggest other response formats. With respect to recall period, participants were asked how far back they/their child could remember. Examples of paper and pencil, computerised or app-based administration modes were given. The aspects explored with participants are shown in more detail in Table 1.

Table 1 Aspects of measure design explored in interviews using COSMIN recommended measurement properties [9]
Full size table

The topic guide was reviewed by the study steering group (healthcare professionals, parents and researchers). Interviews were conducted by LC (experienced children’s palliative care nurse, new to qualitative research), AR (experienced in working with children but new to qualitative research) and DB (experienced qualitative researcher). Interviewers did not have any previous relationship with participants. All interviewers received training and supervision on conducting interviews with children, including communication, and legal and ethical issues. Interviews were audio-recorded, transcribed verbatim and pseudonymised.

2.6 Data Analysis

Transcripts were analysed by LC, DH, AR, DB and HS (all female) using deductive and inductive coding (from the World Health Organisation domains of palliative care [31] and COSMIN taxonomy [9]) [32, 33]. Analysis followed the five steps of framework analysis: familiarisation, constructing a thematic framework, indexing and sorting, charting and mapping/interpretation [32,33,34] using NVivo software (Version 12). All researchers received training on the use of Nvivo and framework analysis. Regular meetings were held to discuss emerging themes and resolve any differences (20% of transcripts were independently coded by two researchers [32]). KB, CES and RH were consulted if needed to resolve discrepancies. Analysis was reviewed by the study steering group throughout the study.

2.7 Ethical Approval

Ethical approval was granted by the Bloomsbury research ethics committee (HRA:19/LO/0033). Participants 16 years old and over provided written informed consent. Those with parental responsibility provided written informed consent for participants < 16 years. Those < 16 years provided written or verbal assent.

3 Results

3.1 Participant Characteristics

Seventy-six interviews were conducted (April 2019–September 2020) with 79 participants: 39 children aged 5–17 years (26 living with a life-limiting condition; 13 healthy siblings) and 40 parents (of children aged 0–17 years). Two sets of parents and one set of siblings were interviewed together. International Classification of Diseases 10th Revision (ICD-10) chapter headings are reported for pseudonymity, as some children had rare conditions. Interviews were carried out face-to-face in a location of the participant’s choosing, with the exception of 13 interviews that were conducted remotely via video call due to the coronavirus disease 2019 (COVID-19) pandemic [35]. No participants required the use of an interpreter. Table 2 shows participant demographic data.

Table 2 Participant demographics
Full size table

3.2 Main Findings

Participants spoke about aspects of PCOM recall period, response format and measure administration that encompassed the COSMIN content validity standards of relevance and comprehensibility. They also discussed aspects of feasibility and acceptability of a PCOM designed to measure health outcomes in children with life-limiting conditions, such as length and number of questions. Table 3 shows the findings of this study mapped onto these themes.

Table 3 Main findings mapped on to COSMIN properties of content validity, feasibility and acceptability [9]
Full size table

3.3 Response Format

Children with life-limiting conditions as young as 8 years old were familiar with the numerical rating scale, and seemed to comprehend how to use this, especially in relation to pain assessment. They were also able to use the scales to report on other symptoms, such as worry and sleep.

“They usually ask me like ‘on a scale of 1 to 10’” (Child, 10 years old, respiratory condition)

Most children found visually appealing response formats more relevant and acceptable, predominantly the 6-point Likert faces scale. Children as young as 5 years old seemed to understand how to use scales anchored with faces. However, a small minority of teenage participants stated that numerical rating scales were more acceptable for them and felt that faces were more appropriate for younger children.

‘[Investigator] I: ‘So, these are different faces, so again the smiley face would be no hurt and then that really sad face, do you know what that would be?

[Participant] P:‘Umm that really hurts, and that one, that would hurt a little bit, that would hurt a little bit as well, and that would hurt a little bit more and that would hurt a whole lot….’ (Child, 5 years old, gastrointestinal condition)

‘I think those are like more my age, and then like the faces could be like for younger kids.’ (Child, 15 years old, gastrointestinal condition)

One child felt that use of the faces scale could lead to ambiguous interpretations about how they felt. There was a concern that one could be experiencing a high level of pain or distress, but that this would not necessarily be reflected in the selected facial expression. This led to the concern that people would think your symptoms were not as bad as they were:

‘Say I felt like…like 0 and I was like this but actually in the inside I’m 10? Mm because erm…sometimes like people could be hurting out of 10 but then people could say, it’s not hurting out of 10 […]…because the guy isn’t crying and like you’re not exactly like the face but then you don’t have to like (makes noise) squeeze out a tear or…’ (Child, 10 years old, gastrointestinal condition)

None of the participants had seen or used the colour block scale before. Only one child found it acceptable for use as it was similar to a computer game they played:

‘I: If you had to choose one?

P: That one.

I: The blocks? Yeah, and why do you like the blocks? Do you know why you like that one more than the others?

P: Because there’s something called number blocks

I: Oh, do you use them at school?

P: No there’s a programme and it…and it…and erm…it has numbers all the way up and it keeps going up and up and I saw one what said, one hundred’

In contrast to children with life-limiting conditions, healthy siblings were less familiar with rating scales and struggled to comprehend them, particularly those under 11 years old. Those older than 11 were more able to use common rating scales.

‘The sad faces are for not important and the angry faces for so not important’ (Sibling, 7 years old, of a child with a neurological condition)

‘May be ask like erm…what’s worrying you on a scale of 1–10 how is…how is this making you feel. Erm, how much are you worrying about this problem?’ (Sibling, 12 years old, of a child with a neurological condition)

Parents and carers were generally in agreement with the children that visually appealing response formats would be more acceptable, relevant and comprehensible than numerical rating scales for the children. Some also expressed that older children and teenagers may find a numerical rating scale more acceptable, with emojis being suggested to anchor scales, as all children were familiar with them. One parent expressed concern about the acceptability of using traffic light colours for scales, as these are often used for behaviour management in schools:

‘They use a lot of that for like autistic children and things for behaviour and at school. At [patient] and [sibling’s] school, you know, their behaviour chart is green, amber and red.’ (Parent of child with cancer)

3.4 Recall Period

Most children with life-limiting conditions and their healthy siblings proposed an acceptable recall period of a few days up to a week. Overall, there was no difference in recall period preference with respect to age, although some older children and teenagers stated that they could only remember the past few days, while others suggested a week or more. Some participants felt they would forget the past week:

‘P: Like today one of the Drs asked how long have I been having my headaches.

I: Ok, and do you find it easy to remember about the last week?

P: No, I forget very easily.’ (Child, 12 years old, gastrointestinal condition)

Some children with life-limiting conditions did not want to reflect back further than a few weeks, wanting to put the past behind them, which suggests a long recall period is less acceptable. The reasons for this included not wanting to be reminded of past periods of ill health, and a desire to have current symptoms and concerns addressed so that the focus could be on undertaking normal childhood activities such as attending school and seeing friends.

‘[E]rm I suppose...the more recent is the better because, I don’t know, I think sometimes people want to put like past things sort of behind them you know and move on. And because things change so much and I think most, the more recent are better because it’s easier to remember and to focus on what you’re going through at that stage rather than things that have happened’ (Child, 14 years old, cancer).

One participant suggested that recalling salient aspects of their illness was easier than remembering usual activities of daily life, suggesting that acceptable and feasible recall periods may differ depending on the question.

‘It depends on like the thing, if it’s like, what I had for dinner last night or like the last night before that, I can’t really remember (laughter), but then like if it’s like what happens when I was ill, I can pretty much remember that exactly’ (Child, 13 years old, gastrointestinal condition).

3.5 Length/Completion Time

There was broad consensus across participants that a short measure, with enough questions to elicit how a child was feeling, but not placing undue burden in terms of completion time, was most feasible. When asked to specify what length would be acceptable for a measure, ten questions was the most frequent response, with optimal completion time ranging from 5 to 15 min. The majority of participants felt that 10 min was an optimal completion time.

‘I: How many questions do you think?

P: Probably like 10?’ (Child, 14 years old, congenital condition)

‘Yeah, I suppose for the kids then something like 10 minutes’ (Mother of child with cancer)

3.6 Administration Mode

Parents overwhelmingly thought that children would find electronic modes of administration more acceptable and feasible because children are familiar with technology.

‘P: What an app that you can log into or something?

I: Yeah, like what do you think would be better for the kids?

P: Well kids are more technological nowadays anyway, so that would probably suit them better.’ (Father of child with a gastrointestinal condition)

Whilst this was the preference of some children, several expressed a strong preference for a paper-based measure or had no preference. Most siblings also found computerised administration modes more acceptable, with a few expressing a preference for pen and paper. The preference for pen and paper seemed to be in part because they thought they would be more likely to have someone with them during measure completion to help if they did not understand what to do:

‘I: And then if we had a questionnaire like that, do you think it would be better to give it to you on a pen and paper or an iPad or a laptop…?

P: A pen and paper’ (Child, 9 years old, gastrointestinal condition)

‘Umm…I wouldn’t really mind if I’m honest. Like writing would be a lot better because sometimes things online, you can’t…you don’t really understand and this is like, if someone’s in front of you they’ll tell you what to do then…’ (Sibling, 11 years old, of child with a congenital condition)

3.7 The Need to have Someone to Talk to in Measurement

Throughout the interviews, children expressed the importance of having someone from the healthcare team to talk to during or after measure completion. There were several reasons cited for this. Some children wanted to be able to clarify potential concerns regarding comprehension and interpretation of questions in a measure. Other reasons included ensuring children were honest in answering questions and to give the child’s healthcare experience a ‘human and compassionate’ feel. Like children with life-limiting conditions, siblings preferred to have someone to talk to about how they were feeling either in addition to choosing response options on a measure or instead of measure completion.

‘I’m thinking like the patient should fill something out but then on top of that, you know have the discussion with the…with the person, the healthcare assistant or healthcare professional…erm because umm…the…when the patients filling out the form, may…the…the patient may not…either may not be erm…like fully honest or…or they may not understand the erm…the question, because that actually happens.’ (Child, 17 years old, gastrointestinal condition)

‘[Y]ou know like, you can’t just substitute the healthcare professional for a robot, you want to kind of have that human feel, so that’s…that’s important because otherwise the patient may not want to say anything.’ (Child, 17 years old, gastrointestinal condition)

‘You could just put it in front of them and ask their opinion and just ask them to circle it and…and ask them if they wanted to expand a bit more about their…how they’re feeling’ (Sibling, 11 years old, of a child with a congenital condition)

4 Discussion

This study provides evidence on the acceptability, feasibility, relevance and comprehensiveness of PCOM design properties for children with life-limiting conditions. We found that children with life-limiting conditions find brief measures more acceptable and feasible to use, and shorter recall period to be acceptable and relevant. Most stated that electronic measures are more acceptable for use, although differences in preferences indicate that measures should also be available in paper and pen format. Additionally, whilst children with life-limiting conditions can comprehend numerical rating scales and use them to report on their health, most find visually appealing response formats using emotive faces more acceptable. Finally, we found that children want to complete a measure alongside a conversation with healthcare professionals about their care needs and priorities.

The findings of this study largely support a recent systematic review on recall period, response scale format and administration mode in healthy children, and those with acute or chronic conditions with a good prognosis [18]. However, contrary to previous investigations, we found that a small number of participants had a strong preference for paper-based measures, indicating that it is important to offer various modes of administration [36,37,38,39,40,41,42]. Notably, this finding also conflicts with parent/carer beliefs that all children would find electronic modes of administration more acceptable, demonstrating the importance of asking children with life-limiting conditions about their own preferences and not relying exclusively on proxy reports.

Our study also highlights that children with life-limiting conditions have a desire to talk about how they are feeling directly with healthcare professionals, in addition to PCOM completion, to ensure their healthcare experience is ‘human and compassionate’. This will enhance acceptability of the measure. Similarly, in adults, PCOMs facilitate patient-centred communication by providing overview and insight and by prompting discussions about topics that are important to patients [43, 44]. Our findings with children need to be considered when PCOMs are implemented into practice within this population, so that the intended goals are achieved and care is focused on the child and family and what is important to them.

Children with life-limiting conditions demonstrated more familiarity with and understanding of common rating scales compared to their healthy counterparts. This was evident in our study where healthy siblings, who were less familiar with commonly used rating scales, often struggled to comprehend them. This is likely to be because children with life-limiting conditions have more exposure to describing their health than other children when being asked about pain and other symptoms. The difference between children with life-limiting conditions and other ill children and/or healthy peers also underscores the need to understand the needs of this population and not rely on outcome measures created for use in other illness populations or healthy children.

Participants expressed variable preferences in recall period, with most stating a short recall period of between a day and a week was most relevant and acceptable. Previous studies have found that recall should be kept to 24–48 h for those under 8 years [18, 45, 46], with those over 8 years being able to reliably recall events from the past 7–14 days [18, 45, 47,48,49]. The difference in our findings may reflect the variability in cognitive ability among children with life-limiting conditions [14]. Some participants suggested that when it comes to recalling salient events regarding health, such as episodes of being more unwell, they can remember further back than they can for details of day-to-day life. Previous studies have reported that children found it easier to remember what had happened between specific events, such as clinic appointments [50]. This may indicate that PCOM questions regarding physical symptoms such as pain may be easier for children to respond to than questions regarding whether they were able to undertake their usual day-to-day activities. However, despite episodes of past ill health being more salient to children with life-limiting conditions, many did not want to reflect that far back. They either did not want to be reminded of a negative experience, did not think it was relevant or wanted to move on and focus on the present. This calls attention to what constitutes respondent burden and the need to address it when developing PCOMs for this population to ensure acceptability in the target population.

4.1 Strengths and Limitations

The strengths of this study include the involvement of children from the age of 5 years old, with a range of life-limiting conditions. We also included the perspectives of healthy siblings and parents. Many studies reporting on this population rely on proxy reports from parents/carers and healthcare professionals, or focus on children with a cancer diagnosis [14]. Our sample size was relatively large in comparison to other studies that include children with life-limiting conditions, and the geographical spread of participant recruitment covered several areas of the UK, across two countries. We were also able to compare the perspectives of children with life-limiting conditions with those of their healthy siblings.

Our study has some limitations. The siblings of children with life-limiting conditions were likely to have had more exposure to conversations regarding healthcare than other children of the same age, so caution should be taken in extrapolating this finding to other healthy children. Although we included children as young as 5 years in our study, relatively few children with life-limiting conditions under 8 years old were recruited, meaning the data presented cover to a greater extent the perspectives of those who were towards the older age limit. One site recruited only children with gastrointestinal diagnoses, reflected in the higher number of participants from this group. There are almost 400 different life-limiting conditions known to affect children, so not all could be included [51]. Our parent sample was predominantly female, which reflects other paediatric palliative care research studies, where fathers are under-represented [52]. No participants were recruited who required an interpreter, and data on ethnicity were not collected. Therefore, our findings may not reflect culturally diverse perspectives. Finally, some interviews with children were short due to difficulty keeping them engaged, or due to illness related fatigue, which further highlights the need for a short, brief PCOM for this population. All participants discussed some aspects of recall period, response format and administration mode in their interviews. However, sometimes this was a small part of the overall interview as these questions were asked at the end. Therefore, the perspectives presented may be from those who were more willing or able to respond to a PCOM.

4.2 Implications for Research and Practice

This study endorses some of the findings of a recent systematic review on recall period, response format and administration mode in predominantly healthy children. Recall periods of PCOMs should be kept short due to the difficulty children have in remembering too far back, and to reduce respondent burden of having to dwell too much on periods of past ill health. This study also provides some additional insights into PCOM development in children with life-limiting conditions [18]. When developing, validating and implementing PCOMs for children with life-limiting conditions, consideration should be given to ensuring that, where possible, they are given the opportunity to discuss their responses with a healthcare professional during or soon after completion. Children need to know that their responses are seen and considered. This supports a child-centred approach whereby children are regarded as active and equal partners in their care [53]. In contrast to findings in healthy children, a choice of electronic or paper and pencil format should be given to those with life-limiting conditions where possible, to ensure acceptability and feasibility of use of PCOMs.

4.3 Next Steps

Further research is required to generate initial versions of the C-POS and demonstrate comprehensiveness, comprehensibility and acceptability using cognitive interviews. This will be followed by psychometric testing.

5 Conclusions

Children with life-limiting conditions are able to describe their health outcomes [22] and can communicate their preferences regarding PCOM design. Incorporating these preferences should improve acceptability of the measure and enhance its uptake in clinical practice [3]. Children with life-limiting conditions expressed a strong desire for the opportunity to be able to discuss their symptoms and care concerns with healthcare professionals alongside PCOM completion and valued the opportunity to report on this. Children’s views and preferences should be included early on and throughout measure development to improve design and enhance valid and reliable self-report.