FormalPara Key Points for Decision Makers

Participants in traditional health state valuation exercises are not specifically told what to consider when they imagine what it is like to live in a particular health state.

This pilot study sought to investigate whether participants consider the impact of the health state on others (e.g. informal carers) in a traditional exercise, and whether their valuations would be altered when told that they would need a carer living in a certain health state.

The results of this study indicate that around one-third of individuals consider being a burden on others in the traditional exercise, and providing contextual information about carers’ quality of life may have an impact on individual valuations.

1 Introduction

The impact of illness extends beyond that of the individual patient. Many patients, particularly those with chronic conditions, receive informal care that is provided by family and friends. Not only are there significant opportunity costs associated with the provision of informal care, but there can also be significant health impacts for informal carers [1]. Despite this, it has recently been highlighted that these impacts are rarely considered in economic evaluations of new treatments by health technology assessment agencies such as the National Institute for Health and Care Excellence (NICE) [2, 3]. This is in spite of evidence suggesting that informal care can have a substantial impact on the outcomes of cost-utility analyses (CUAs) when it is included [3, 4]. On the other hand, it has been suggested that the inclusion of carer quality of life (QoL) may have an adverse impact on equity, particularly for those with limited access to informal care [5].

While carer QoL has not been routinely considered in NICE appraisals [2], researchers have applied a range of instruments to measure carer QoL. As with patients, carer QoL can be measured using EQ-5D (NICE’s preferred measure) [6, 7], as well as other generic and carer-specific measures [8]. The EQ-5D is a generic preference-accompanied measure [9] that covers five dimensions of health: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. There are two versions for use in adult populations, one with three severity levels for each dimension (EQ-5D-3L) and one with five levels (EQ-5D-5L). Accompanying value sets allow responses to be converted into utilities for use in quality-adjusted life-year (QALY) calculations, where a value of zero is equivalent to being dead, a value of one is equivalent to full health, and negative values indicate states that are worse than being dead.

To generate a value set for a QoL measure such as EQ-5D, a health state valuation exercise using a choice-based stated preference method is usually conducted. For example, the EQ-5D-3L value set for the UK was valued using time trade-off (TTO) methodology [10]. TTO requires participants to indicate the amount of life (in full health) that they would be willing to trade to avoid living in an imperfect health state [11]. During a TTO exercise, participants (typically members of the general population rather than patients) must imagine living in a range of imperfect health states, with no specific guidance from the interviewer on what impact the health states might have on their lives.

A small number of studies have set out to explore individuals’ thought processes when completing valuation tasks and found that participants often considered being a burden on others [12,13,14,15]. Robinson et al. [14] conducted 43 TTO interviews with participants from the EQ-5D-3L UK valuation study [10] and found that the majority of participants (n = 27; 63%) mentioned being a burden on others. This finding was consistent across different age groups. Similarly, Baker and Robinson [12] conducted 28 standard gamble interviews and found that many participants considered factors such as being a burden on carers and family when partaking in the valuation task. In another study, van der Pol and Shiell [13] aimed to identify whether recent mothers (n = 30) in Canada aim for a target life expectancy in TTO interviews. While they did not find support for their hypothesis, some participants did express concern over being a burden on their children or the rest of their family. More recently, Karimi et al. [15] conducted 21 ‘think aloud’ interviews with TTO and discrete choice experiment (DCE) tasks in the UK. Being a burden on others was one of the six consequences that was most frequently mentioned by participants when they were imagining themselves in the health states being valued.

While there is qualitative evidence that individuals think about the impact of being in a health state on others when taking part in valuation exercises, there is less evidence on the extent to which this might be reflected in their (quantitative) responses. Krol et al. [16] explored this by conducting an online TTO exercise with four different arms. Arm one was a conventional exercise, arm two asked respondents explicitly to consider the effects of their ill-health on loved ones, and arm four provided respondents with additional contextual information that indicated that the health state would result in some reliance on loved ones without explicitly instructing respondents to consider the effects in their valuations. The authors found that just over half of respondents spontaneously considered altruistic preferences in arm one. They concluded that health state valuations were higher when participants had a desire to live longer for loved ones (referred to as longevity altruism), and lower when participants were concerned about being a burden on loved ones (referred to as quality-of-life altruism). When instructions were provided to explicitly consider altruistic preferences, as in arm two, they found that this led to a small domination of quality-of-life altruism over longevity altruism. Finally, although a slightly higher percentage of respondents reported considering the effects on others in arm four compared with arm one, the valuations did not differ significantly between these arms.

Given that informal care is typically provided by loved ones, and that individuals might consider the impact on loved ones in valuation tasks, it could be argued that including separate carer health effects in economic evaluations may lead to some level of ‘double counting’ [16]. This paper reports the methods and findings of a pilot TTO valuation study that aimed to explore the feasibility of a face-to-face approach designed to investigate whether utilities differ when respondents are informed that the health state will require them to have a carer.

2 Methods

2.1 Study Overview

Face-to-face interviews were conducted involving two different TTO exercises. The first exercise followed a typical approach to value three EQ-5D-5L health states (the ‘traditional’ exercise). In the second exercise, participants valued the same health states again but were told to consider the fact that they would need an informal carer when living in the health state and were provided with the carer’s EQ-5D-5L health state (the ‘combined’ exercise). The intention was not to jointly value the two health states but for the participant to consider the existence of the carer and the carer’s health state as additional contextual information when valuing the health states. If participants did not provide different values between the two tasks, it may be an indication that they considered the impact of the health state on others (e.g. informal carers) when responding to the ‘traditional’ exercise. Qualitative data were also collected to corroborate this assertion. Similar study designs have been employed elsewhere, albeit in the context of productivity losses [17, 18].

The UK Health Research Authority decision tool indicated that approval from an NHS Research Ethics Committee (REC) was not required. Nonetheless, as good research practice, we sought an ethics review from an independent reviewer working under the auspices of the Association of Research Managers and Administrators. The reviewer, David Carpenter, has extensive experience in research ethics, including being a long-standing NHS REC chair, and provided approval for the study in November 2019.

The remainder of this section provides more detailed information about the design and implementation of the interviews, as well as the approach taken to analyse the data. A completed TTO study checklist (from Attema et al. [19]) that provides a standardised summary can be found in the electronic supplementary material (ESM).

2.2 Time Trade-Off (TTO) Interviews

2.2.1 Structure of the Interviews

Figure 1 illustrates the overall structure of the interviews. The interview began with an introduction and background questions. After this, the health states were introduced and ranked in order of preference by the respondent (alongside being dead). Participants were then introduced to TTO methodology and completed a practice task. The traditional exercise was then completed, followed by the combined exercise, with questions asked after each exercise about what they took into consideration when responding. Finally, participants were asked some debriefing questions about the survey. The interviewer script and data collection materials can be found in the ESM.

Fig. 1
figure 1

Structure of the TTO interviews. TTO time trade-off

Full size image

2.2.2 Health State Selection

Given that the combined exercise required combinations of health states (one for the participant to imagine living in and one for the carer as additional context), we sought to find realistic combinations. EQ-5D-5L health states for patient-carer dyads were identified from a dataset from an earlier study conducted in the context of meningitis [20, 21]. We considered it important to select dyads that contained health states with different severity levels. The ‘level sum score’ (LSS), which is the sum of the levels on each dimension of the EQ-5D-5L, was used to determine the approximate severity of the health states [22]. LSS ranges from 5 for the EQ-5D-5L health state 11111 (1+1+1+1+1; no problems on any dimension) to 25 for 55555 (5+5+5+5+5; worst possible response). The LSS is a crude summary score, but it was helpful in this context to remove mild individual health states (i.e. those with a low LSS), in case participants would not find it plausible that individuals in these health states would require support from carers. Three individual health states (with accompanying carer health states) were selected, as this would result in a total of six valuations (three traditional plus three combined) being made in the hour-long interviews, which seemed feasible compared with other studies [10, 23,24,25]. The final health states selected are set out in Table 1.

Table 1 Health states used in the TTO exercises
Full size table

2.2.3 The TTO Approach

The composite TTO approach was used in this study [26], which is the variant used in current EQ-5D-5L and EQ-5D-Y-3L valuation protocols [27, 28]. In this approach, states better than dead are valued using the conventional TTO method and states worse than dead are valued using the lead-time TTO method. A standard time horizon of 10 years was used in the tasks, with a lead-time of 10 years in the lead-time TTO. We aimed to closely replicate the EQ-VT protocol, but used physical time boards rather than a digital version [28].

The two TTO exercises in our study did not differ significantly in their methodological approach. The only difference was that in the combined exercise, the carer's health state was placed to the side of the individual health state being valued on the time board to provide additional context. Interviewers instructed participants that their impaired condition means that they require an informal carer, communicated as an individual who has a personal relationship with the participant and provides unpaid day-to-day support, and who is not in full health either (no reason for their ill health was provided). No further information about the carer was provided.

The order in which the health states were valued was based on rankings of the individual health states that were obtained prior to the TTO exercise, with the states valued from ‘best’ to ‘worst’. After each TTO exercise, participants were asked to describe the factors that they took into consideration when making their choices, initially without any prompts from the interviewer. After this had been asked again following the combined exercise, participants were also directly questioned about the type of carer that they were imagining and whether their answers would have differed if the carer had been in full health.

2.2.4 Interview Implementation

Four interviewers conducted the interviews for the study, all of whom were employed within health economics, health outcomes and medical writing teams at Roche Products Ltd (Roche). All interviewers received training on composite TTO and the overall interview process by two experienced researchers in a face-to-face workshop.

A convenience sample with a target sample size of 30 was sought for the pilot study, consisting of employees working within the Roche premises in Welwyn Garden City, England. Participants were not eligible if they were not a fluent and literate English speaker; had prior familiarity with health state valuation exercises and their methods; or were previously/currently employed in roles related to health economics or outcomes research. The study was advertised internally and the hour-long interviews were organised by a dedicated research coordinator under the supervision of the lead investigator (both also employed by Roche). Participants did not receive any incentives to take part in an interview. Interviews took place between 4 December 2019 and 6 February 2020.

2.3 Data Analysis

Participant characteristics were summarised using descriptive statistics. Responses to the TTO exercises were converted to utilities and summarised descriptively. Mean differences in utilities between the two TTO exercises were calculated and paired t tests were conducted to test the statistical significance of any differences. The individual differences in valuations for each health state were also illustrated visually using spaghetti plots. Analysis of qualitative responses followed a content analysis framework, similar to the approach taken by Shah et al. [29, 30]. Responses (interviewer notes) related to factors for consideration during the tasks (when unprompted) and the type of carer considered in the combined exercise were coded by two members of the study team independently, with any disagreements discussed and resolved together, and were summarised descriptively.

3 Results

3.1 Participant Characteristics

Thirty-three participants took part and their characteristics are summarised in Table 2. The mean age of participants was 43 years, and a large majority of participants were female (n = 24; 73%), had a university degree (n = 28; 85%), and worked full-time (n = 32; 97%). A slim majority of participants reported having at least one dependent below the age of 18 years (n = 17; 52%) and relatively few had dependents above the age of 18 years (n = 5; 15%).

Table 2 Participant characteristics
Full size table

3.2 Health State Utilities

Average utilities for each health state in each exercise are set out in Table 3. Health state A was considered better than dead in all but two cases (both in the combined exercise), with mean utilities of 0.794 in the traditional exercise and 0.717 in the combined exercise. Despite the severity of health states B and C, these were considered better than dead in the majority of cases, irrespective of the exercise type. Nonetheless, mean utilities for these two health states were negative in all but one instance (0.071 for health state B in the combined exercise). Overall, the differences in the proportions of participants considering a health state as better or worse than dead did not differ substantially between the exercises.

Table 3 Average health state utilities by task and mean differences
Full size table

Table 3 also contains the mean differences between utilities in the two TTO exercises, alongside results from the paired t-tests. Mean differences were all around 0.07 in magnitude but varied in direction. Statistical significance should be interpreted with caution since this pilot, based on a convenience sample, was not powered to detect any meaningful differences. That said, if a 10% threshold is used as a sign of increasing evidence against the null hypothesis of no difference [31], then only health state A may provide some evidence of a detrimental carer effect (p value 0.101).

The spaghetti plots in Fig. 2 highlight the variation in utilities at the individual level. Red lines indicate that the utility from the traditional exercise was larger than the utility from the combined exercise, and grey lines indicate the opposite (including no change). It is clear from Fig. 2 that there was a lot of variation between participants, both in the direction and magnitude of the differences, potentially explaining the absence of effect at the group level (Table 3). Although there may be some evidence of a difference between exercises for health state A, Fig. 2 highlights a potential ceiling effect. Thus, overall, the results do not provide evidence of consistent differences in utilities between the two exercises.

Fig. 2
figure 2

Spaghetti plots of individual-level responses to each TTO exercise. Red lines indicate that the valuation in the combined exercise was lower than in the traditional exercise, and grey lines indicate that the valuation in the traditional exercise was lower than or equal to valuation in the combined exercise. TTO time trade-off, TE traditional exercise, CE combined exercise

Full size image

3.3 Factors Taken into Consideration During the TTO Exercises

The analysis of responses to the question about the factors taken into consideration when valuing the health states (prior to any interviewer prompting) focused on comments relating to independence and impacts on other individuals. Table 4 sets out the frequency with which different factors were mentioned by participants and subsequently recorded by the interviewers in their notes. A total of 13 different codes were identified under five broader headings, of which one heading (carer-specific) was only relevant for responses following the combined exercise. The codes under the ‘burden on others’ heading are split by the identity of the individual(s) for whom the participant is concerned about and by the type of concern. Descriptions and examples for each code are provided in the ESM.

Table 4 Factors taken into consideration during the TTO exercises
Full size table

In the traditional exercise, 36% (n = 12) of participants expressed concern about the burden on others. The exact nature of the burden was rarely expressed (or noted down by interviewers), although the financial concerns raised by some participants illustrate that this is not always in relation to care requirements. Participants generally referenced family or loved ones when discussing being a burden on others, although some referred to an unspecified individual or carer more generally. A concern about (loss of) independence, which we considered to be different to a concern about being a burden, was also expressed by several participants in the traditional exercise (n = 10; 30%). In the combined exercise, the proportion of participants expressing a concern for the burden on others increased to 76% (n = 25). Additionally, 45% (n = 15) expressed a concern about the carer’s well-being (given the carer’s health state) and 15% (n = 5) expressed a concern over the carer’s ability to provide care while living in their health state.

Table 5 details participants’ responses when directly asked about who they were thinking about in the role of the carer when completing the combined exercise. There was a wide range of responses (including multiple responses for some individuals), and five participants (15%) considered a professional carer. Overall, most participants stated that they thought about their significant other in the carer role (79%; n = 26).

Table 5 Presumed identity of the carer in the combined TTO exercise
Full size table

In addition, 67% (n = 22) of participants stated that their responses to the combined exercise would have differed had the carer been in full health.

3.4 Feedback

Overall, 45% of participants rated the traditional TTO exercise as ‘difficult’ (n = 15; no participants rated it ‘very difficult’). In contrast, 61% of participants rated the combined TTO exercise as ‘difficult’ or ‘very difficult’ (n = 20; of which n = 5 rated the exercise as ‘very difficult’).

4 Discussion

4.1 Summary of Findings

In this study, we piloted an approach to explore whether responses to TTO tasks differ when participants are provided with additional contextual information around their need for a carer when living in the health state being valued, as well as the carer’s health state. While participants did rate the combined exercise as more difficult than the traditional exercise, the combined exercise appeared to be feasible to conduct in practice. The mean utilities had face validity in that health state A was preferred to health state B, which was preferred to health state C (irrespective of the exercise). Mean utilities for each health state in the traditional exercise were higher than those observed in the EQ-5D-5L crosswalk value set for the UK [32].

Mean differences in utilities between the two exercises were generally larger in magnitude than many of the estimated minimally important differences for EQ-5D-5L [33, 34], suggesting that differences of this magnitude could be meaningful. However, there was inconsistency in the differences in valuations between exercises (different directions), which meant that there was no strong evidence for a consistent carer effect at a group level. Nonetheless, it cannot be concluded that participants were valuing health states very similarly between the two exercises, as illustrated by the heterogeneous individual response trajectories in Fig. 2. There was also a considerable split between the proportion of individuals who considered health states B and C to be better or worse than dead in both exercises, which added to the noise in the data.

It was expected that many participants would provide lower values in the combined exercise due to a concern for the carer, i.e. the quality-of-life altruism effect observed by Krol et al. [16]. However, a longevity altruism effect may also have occurred with some participants, and some participants may also have been comforted by the confirmation that they would have a carer, both of which could work in the opposite direction. It was also found that the well-being of the carer in the combined exercise was a concern for many participants (n = 15; 45%). The carer's health state in the combined exercise is likely to have differed to the actual health state of the individual(s) that participants imagined for the carer role (Table 5). These differences in health status could have exacerbated either of the altruism effects, affecting valuations in either direction. Furthermore, TTO is known to be subject to learning effects, which may have had a greater impact on the results of the combined exercise relative to the traditional exercise, as the former was always conducted after the latter [35]. This could also have affected results in either direction.

This broad mixture of potential impacts may go some way in explaining the variation in our results, and highlights the challenges associated with identifying the impact of providing contextual information on valuations. Thus, while the task appeared feasible to conduct in practice, identifying the specific impact of the provision of additional contextual information using this approach might not be feasible without substantial adjustments being made to the approach (see Sect. 4.3).

4.2 Study Limitations

This pilot study had several limitations. First, the study sample size was small and the study was underpowered for the effect size observed. With only 33 participants, the study would be powered to pick up mean differences of a magnitude of around 0.2, which is larger than the observed mean differences of around 0.07. Second, convenience samples are susceptible to selection effects (explaining our relatively homogenous sample) and our results are likely to have differed if we had recruited a representative sample of the general population. Third, we aimed to follow good practice when conducting the study [19, 25], with a particular focus on EQ-VT protocol [28]. However, we did not employ the EQ-VT digital aid and instead used physical props, given the flexibility required for the combined exercise. We therefore did not benefit from the interactive features of the computer-based tool and the feedback module that has been shown to improve data quality [36]. However, we sought to maximise data quality by providing all interviewers with the same initial training, which was followed up with one-on-one coaching sessions, and by including a warm-up exercise in the interviews. Fourth, while it may be typical to estimate mean utilities using Tobit models that control for sociodemographic characteristics, our analysis focused on direct comparisons of mean differences using paired t-tests. We felt this was justified by our limited sample size, the lack of variation in sociodemographic characteristics, and the need for a panel set up to incorporate observations from both exercises; however, our analysis may be limited as a result. Fifth, while the carer was defined as an informal carer at the beginning of the combined exercise, five participants stated that they considered professional (formal) carers, suggesting that not all instructions were followed precisely by all participants. Finally, another limitation was that the interviews were not audio recorded and it was therefore not possible to obtain full transcripts. Transcripts would have provided richer data for the qualitative analysis.

4.3 Future Research

While our pilot study has shown that providing contextual information in health state valuation exercises is feasible, it has also illustrated the potential difficulties in understanding the differences in quantitative results. Nonetheless, further research in this area would be beneficial and this pilot study may provide useful insights for such studies.

Future studies of this nature may benefit from taking a simpler approach to the present study, potentially controlling for competing effects such as mentioning the need for a carer but not specifying their health state (or simplifying the health state provided). Furthermore, while this pilot study did directly refer to carers as opposed to loved ones more generally, it may have been beneficial to have explicitly instructed participants to consider the information in their valuations as this may have been more impactful than simply providing the information, as the results of the study by Krol et al. [16] suggest.

Many participants (n = 22; 67%) stated that their responses would have differed had the carer been in full health, suggesting that this information was relevant to the valuations being made. However, informal feedback from interviewers suggested that some participants struggled to take on board multiple EQ-5D-5L health states at the same time, suggesting that the inclusion of this information increased the difficulty of the exercise. An alternative approach that would enable the impact of the carer health states to be explored further would be to have two combined exercises, where only the carer’s existence is described in the first exercise and the carer’s health state is introduced in the second exercise. Furthermore, the carer's health state could be described using a carer-specific instrument such as CarerQol to reduce the focus on EQ-5D-5L and provide a more relevant description [37].

Future studies may also benefit from collecting more data and recording interviews for transcripts. Directly asking participants to reflect on whether the information about needing a carer is comforting or concerning overall would be useful. It would also be helpful to identify participants who may be unwilling to trade life-years due to longevity altruism, as this has an opposite effect to quality-of-life altruism, but occurs as a result of a similar thought process (i.e. considering the impact on others) [16].

Additionally, future studies may also benefit from directly asking participants whether they believe that the implications of their responses are consistent with their beliefs. In some interviews, interviewers noticed contradictions in the responses provided by participants. Using alternative methods that have a more deliberative focus, or implementing direct validity tests into the study design, could be more informative and result in better-quality preference data [38, 39]. It may also be worth considering that a fully qualitative approach could be superior to a quantitative or mixed-methods approach in this area, given the wide range of factors that may influence valuations and the lower sample size requirements that these require.

5 Conclusion

This study sought to explore whether health state valuations differed when contextual information about the need for a carer (and the carer’s health state) was provided, and to gauge the extent to which participants implicitly consider the impact of health states on others. While the mean differences in utilities were not statistically significant between exercises, our results show a high level of individual variation in responses, in different directions, suggesting that the inclusion of such information may have an impact. Further research is required to better understand this variation and the results of this pilot study could be useful in informing the design of such studies in future.