Skip to main content

Impact of Including Carer Information in Time Trade-Off Tasks: Results from a Pilot Study



Carer quality of life (QoL) can be included in economic evaluations and captured using EQ-5D. Traditional valuation tasks require participants to imagine living in a health state for a number of years, without being told what to consider. This pilot study sought to investigate whether participants implicitly consider the impact of the health state on others, and the extent to which this may impact health state valuations.


Composite time trade-off (TTO) interviews were conducted with a convenience sample. Each interview included a ‘traditional’ TTO exercise to value three health states, and a ‘combined’ TTO exercise, where participants valued the same health states again, having been informed that they would require a carer living in a particular health state. Qualitative feedback was collected after each exercise. Paired t-test comparisons of the utilities elicited in each exercise were made.


Thirty-three participants enrolled in the pilot. Mean differences between exercises were not statistically significant and differed in direction, although considerable heterogeneity was observed in individual response trajectories. Overall, 36% (n = 12) of participants expressed an unprompted concern about being a burden on others in the traditional exercise, and 67% (n = 22) of participants would have responded differently had the carer been in full health in the combined exercise.


Providing contextual information about carers may impact valuations. Further research is required to better understand the reasons behind the variation in individual response trajectories observed in this pilot study. The insights from this study may be useful for informing the design of related future studies.

FormalPara Key Points for Decision Makers
Participants in traditional health state valuation exercises are not specifically told what to consider when they imagine what it is like to live in a particular health state.
This pilot study sought to investigate whether participants consider the impact of the health state on others (e.g. informal carers) in a traditional exercise, and whether their valuations would be altered when told that they would need a carer living in a certain health state.
The results of this study indicate that around one-third of individuals consider being a burden on others in the traditional exercise, and providing contextual information about carers’ quality of life may have an impact on individual valuations.


The impact of illness extends beyond that of the individual patient. Many patients, particularly those with chronic conditions, receive informal care that is provided by family and friends. Not only are there significant opportunity costs associated with the provision of informal care, but there can also be significant health impacts for informal carers [1]. Despite this, it has recently been highlighted that these impacts are rarely considered in economic evaluations of new treatments by health technology assessment agencies such as the National Institute for Health and Care Excellence (NICE) [2, 3]. This is in spite of evidence suggesting that informal care can have a substantial impact on the outcomes of cost-utility analyses (CUAs) when it is included [3, 4]. On the other hand, it has been suggested that the inclusion of carer quality of life (QoL) may have an adverse impact on equity, particularly for those with limited access to informal care [5].

While carer QoL has not been routinely considered in NICE appraisals [2], researchers have applied a range of instruments to measure carer QoL. As with patients, carer QoL can be measured using EQ-5D (NICE’s preferred measure) [6, 7], as well as other generic and carer-specific measures [8]. The EQ-5D is a generic preference-accompanied measure [9] that covers five dimensions of health: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. There are two versions for use in adult populations, one with three severity levels for each dimension (EQ-5D-3L) and one with five levels (EQ-5D-5L). Accompanying value sets allow responses to be converted into utilities for use in quality-adjusted life-year (QALY) calculations, where a value of zero is equivalent to being dead, a value of one is equivalent to full health, and negative values indicate states that are worse than being dead.

To generate a value set for a QoL measure such as EQ-5D, a health state valuation exercise using a choice-based stated preference method is usually conducted. For example, the EQ-5D-3L value set for the UK was valued using time trade-off (TTO) methodology [10]. TTO requires participants to indicate the amount of life (in full health) that they would be willing to trade to avoid living in an imperfect health state [11]. During a TTO exercise, participants (typically members of the general population rather than patients) must imagine living in a range of imperfect health states, with no specific guidance from the interviewer on what impact the health states might have on their lives.

A small number of studies have set out to explore individuals’ thought processes when completing valuation tasks and found that participants often considered being a burden on others [12,13,14,15]. Robinson et al. [14] conducted 43 TTO interviews with participants from the EQ-5D-3L UK valuation study [10] and found that the majority of participants (n = 27; 63%) mentioned being a burden on others. This finding was consistent across different age groups. Similarly, Baker and Robinson [12] conducted 28 standard gamble interviews and found that many participants considered factors such as being a burden on carers and family when partaking in the valuation task. In another study, van der Pol and Shiell [13] aimed to identify whether recent mothers (n = 30) in Canada aim for a target life expectancy in TTO interviews. While they did not find support for their hypothesis, some participants did express concern over being a burden on their children or the rest of their family. More recently, Karimi et al. [15] conducted 21 ‘think aloud’ interviews with TTO and discrete choice experiment (DCE) tasks in the UK. Being a burden on others was one of the six consequences that was most frequently mentioned by participants when they were imagining themselves in the health states being valued.

While there is qualitative evidence that individuals think about the impact of being in a health state on others when taking part in valuation exercises, there is less evidence on the extent to which this might be reflected in their (quantitative) responses. Krol et al. [16] explored this by conducting an online TTO exercise with four different arms. Arm one was a conventional exercise, arm two asked respondents explicitly to consider the effects of their ill-health on loved ones, and arm four provided respondents with additional contextual information that indicated that the health state would result in some reliance on loved ones without explicitly instructing respondents to consider the effects in their valuations. The authors found that just over half of respondents spontaneously considered altruistic preferences in arm one. They concluded that health state valuations were higher when participants had a desire to live longer for loved ones (referred to as longevity altruism), and lower when participants were concerned about being a burden on loved ones (referred to as quality-of-life altruism). When instructions were provided to explicitly consider altruistic preferences, as in arm two, they found that this led to a small domination of quality-of-life altruism over longevity altruism. Finally, although a slightly higher percentage of respondents reported considering the effects on others in arm four compared with arm one, the valuations did not differ significantly between these arms.

Given that informal care is typically provided by loved ones, and that individuals might consider the impact on loved ones in valuation tasks, it could be argued that including separate carer health effects in economic evaluations may lead to some level of ‘double counting’ [16]. This paper reports the methods and findings of a pilot TTO valuation study that aimed to explore the feasibility of a face-to-face approach designed to investigate whether utilities differ when respondents are informed that the health state will require them to have a carer.


Study Overview

Face-to-face interviews were conducted involving two different TTO exercises. The first exercise followed a typical approach to value three EQ-5D-5L health states (the ‘traditional’ exercise). In the second exercise, participants valued the same health states again but were told to consider the fact that they would need an informal carer when living in the health state and were provided with the carer’s EQ-5D-5L health state (the ‘combined’ exercise). The intention was not to jointly value the two health states but for the participant to consider the existence of the carer and the carer’s health state as additional contextual information when valuing the health states. If participants did not provide different values between the two tasks, it may be an indication that they considered the impact of the health state on others (e.g. informal carers) when responding to the ‘traditional’ exercise. Qualitative data were also collected to corroborate this assertion. Similar study designs have been employed elsewhere, albeit in the context of productivity losses [17, 18].

The UK Health Research Authority decision tool indicated that approval from an NHS Research Ethics Committee (REC) was not required. Nonetheless, as good research practice, we sought an ethics review from an independent reviewer working under the auspices of the Association of Research Managers and Administrators. The reviewer, David Carpenter, has extensive experience in research ethics, including being a long-standing NHS REC chair, and provided approval for the study in November 2019.

The remainder of this section provides more detailed information about the design and implementation of the interviews, as well as the approach taken to analyse the data. A completed TTO study checklist (from Attema et al. [19]) that provides a standardised summary can be found in the electronic supplementary material (ESM).

Time Trade-Off (TTO) Interviews

Structure of the Interviews

Figure 1 illustrates the overall structure of the interviews. The interview began with an introduction and background questions. After this, the health states were introduced and ranked in order of preference by the respondent (alongside being dead). Participants were then introduced to TTO methodology and completed a practice task. The traditional exercise was then completed, followed by the combined exercise, with questions asked after each exercise about what they took into consideration when responding. Finally, participants were asked some debriefing questions about the survey. The interviewer script and data collection materials can be found in the ESM.

Fig. 1
figure 1

Structure of the TTO interviews. TTO time trade-off

Health State Selection

Given that the combined exercise required combinations of health states (one for the participant to imagine living in and one for the carer as additional context), we sought to find realistic combinations. EQ-5D-5L health states for patient-carer dyads were identified from a dataset from an earlier study conducted in the context of meningitis [20, 21]. We considered it important to select dyads that contained health states with different severity levels. The ‘level sum score’ (LSS), which is the sum of the levels on each dimension of the EQ-5D-5L, was used to determine the approximate severity of the health states [22]. LSS ranges from 5 for the EQ-5D-5L health state 11111 (1+1+1+1+1; no problems on any dimension) to 25 for 55555 (5+5+5+5+5; worst possible response). The LSS is a crude summary score, but it was helpful in this context to remove mild individual health states (i.e. those with a low LSS), in case participants would not find it plausible that individuals in these health states would require support from carers. Three individual health states (with accompanying carer health states) were selected, as this would result in a total of six valuations (three traditional plus three combined) being made in the hour-long interviews, which seemed feasible compared with other studies [10, 23,24,25]. The final health states selected are set out in Table 1.

Table 1 Health states used in the TTO exercises

The TTO Approach

The composite TTO approach was used in this study [26], which is the variant used in current EQ-5D-5L and EQ-5D-Y-3L valuation protocols [27, 28]. In this approach, states better than dead are valued using the conventional TTO method and states worse than dead are valued using the lead-time TTO method. A standard time horizon of 10 years was used in the tasks, with a lead-time of 10 years in the lead-time TTO. We aimed to closely replicate the EQ-VT protocol, but used physical time boards rather than a digital version [28].

The two TTO exercises in our study did not differ significantly in their methodological approach. The only difference was that in the combined exercise, the carer's health state was placed to the side of the individual health state being valued on the time board to provide additional context. Interviewers instructed participants that their impaired condition means that they require an informal carer, communicated as an individual who has a personal relationship with the participant and provides unpaid day-to-day support, and who is not in full health either (no reason for their ill health was provided). No further information about the carer was provided.

The order in which the health states were valued was based on rankings of the individual health states that were obtained prior to the TTO exercise, with the states valued from ‘best’ to ‘worst’. After each TTO exercise, participants were asked to describe the factors that they took into consideration when making their choices, initially without any prompts from the interviewer. After this had been asked again following the combined exercise, participants were also directly questioned about the type of carer that they were imagining and whether their answers would have differed if the carer had been in full health.

Interview Implementation

Four interviewers conducted the interviews for the study, all of whom were employed within health economics, health outcomes and medical writing teams at Roche Products Ltd (Roche). All interviewers received training on composite TTO and the overall interview process by two experienced researchers in a face-to-face workshop.

A convenience sample with a target sample size of 30 was sought for the pilot study, consisting of employees working within the Roche premises in Welwyn Garden City, England. Participants were not eligible if they were not a fluent and literate English speaker; had prior familiarity with health state valuation exercises and their methods; or were previously/currently employed in roles related to health economics or outcomes research. The study was advertised internally and the hour-long interviews were organised by a dedicated research coordinator under the supervision of the lead investigator (both also employed by Roche). Participants did not receive any incentives to take part in an interview. Interviews took place between 4 December 2019 and 6 February 2020.

Data Analysis

Participant characteristics were summarised using descriptive statistics. Responses to the TTO exercises were converted to utilities and summarised descriptively. Mean differences in utilities between the two TTO exercises were calculated and paired t tests were conducted to test the statistical significance of any differences. The individual differences in valuations for each health state were also illustrated visually using spaghetti plots. Analysis of qualitative responses followed a content analysis framework, similar to the approach taken by Shah et al. [29, 30]. Responses (interviewer notes) related to factors for consideration during the tasks (when unprompted) and the type of carer considered in the combined exercise were coded by two members of the study team independently, with any disagreements discussed and resolved together, and were summarised descriptively.


Participant Characteristics

Thirty-three participants took part and their characteristics are summarised in Table 2. The mean age of participants was 43 years, and a large majority of participants were female (n = 24; 73%), had a university degree (n = 28; 85%), and worked full-time (n = 32; 97%). A slim majority of participants reported having at least one dependent below the age of 18 years (n = 17; 52%) and relatively few had dependents above the age of 18 years (n = 5; 15%).

Table 2 Participant characteristics

Health State Utilities

Average utilities for each health state in each exercise are set out in Table 3. Health state A was considered better than dead in all but two cases (both in the combined exercise), with mean utilities of 0.794 in the traditional exercise and 0.717 in the combined exercise. Despite the severity of health states B and C, these were considered better than dead in the majority of cases, irrespective of the exercise type. Nonetheless, mean utilities for these two health states were negative in all but one instance (0.071 for health state B in the combined exercise). Overall, the differences in the proportions of participants considering a health state as better or worse than dead did not differ substantially between the exercises.

Table 3 Average health state utilities by task and mean differences

Table 3 also contains the mean differences between utilities in the two TTO exercises, alongside results from the paired t-tests. Mean differences were all around 0.07 in magnitude but varied in direction. Statistical significance should be interpreted with caution since this pilot, based on a convenience sample, was not powered to detect any meaningful differences. That said, if a 10% threshold is used as a sign of increasing evidence against the null hypothesis of no difference [31], then only health state A may provide some evidence of a detrimental carer effect (p value 0.101).

The spaghetti plots in Fig. 2 highlight the variation in utilities at the individual level. Red lines indicate that the utility from the traditional exercise was larger than the utility from the combined exercise, and grey lines indicate the opposite (including no change). It is clear from Fig. 2 that there was a lot of variation between participants, both in the direction and magnitude of the differences, potentially explaining the absence of effect at the group level (Table 3). Although there may be some evidence of a difference between exercises for health state A, Fig. 2 highlights a potential ceiling effect. Thus, overall, the results do not provide evidence of consistent differences in utilities between the two exercises.

Fig. 2
figure 2

Spaghetti plots of individual-level responses to each TTO exercise. Red lines indicate that the valuation in the combined exercise was lower than in the traditional exercise, and grey lines indicate that the valuation in the traditional exercise was lower than or equal to valuation in the combined exercise. TTO time trade-off, TE traditional exercise, CE combined exercise

Factors Taken into Consideration During the TTO Exercises

The analysis of responses to the question about the factors taken into consideration when valuing the health states (prior to any interviewer prompting) focused on comments relating to independence and impacts on other individuals. Table 4 sets out the frequency with which different factors were mentioned by participants and subsequently recorded by the interviewers in their notes. A total of 13 different codes were identified under five broader headings, of which one heading (carer-specific) was only relevant for responses following the combined exercise. The codes under the ‘burden on others’ heading are split by the identity of the individual(s) for whom the participant is concerned about and by the type of concern. Descriptions and examples for each code are provided in the ESM.

Table 4 Factors taken into consideration during the TTO exercises

In the traditional exercise, 36% (n = 12) of participants expressed concern about the burden on others. The exact nature of the burden was rarely expressed (or noted down by interviewers), although the financial concerns raised by some participants illustrate that this is not always in relation to care requirements. Participants generally referenced family or loved ones when discussing being a burden on others, although some referred to an unspecified individual or carer more generally. A concern about (loss of) independence, which we considered to be different to a concern about being a burden, was also expressed by several participants in the traditional exercise (n = 10; 30%). In the combined exercise, the proportion of participants expressing a concern for the burden on others increased to 76% (n = 25). Additionally, 45% (n = 15) expressed a concern about the carer’s well-being (given the carer’s health state) and 15% (n = 5) expressed a concern over the carer’s ability to provide care while living in their health state.

Table 5 details participants’ responses when directly asked about who they were thinking about in the role of the carer when completing the combined exercise. There was a wide range of responses (including multiple responses for some individuals), and five participants (15%) considered a professional carer. Overall, most participants stated that they thought about their significant other in the carer role (79%; n = 26).

Table 5 Presumed identity of the carer in the combined TTO exercise

In addition, 67% (n = 22) of participants stated that their responses to the combined exercise would have differed had the carer been in full health.


Overall, 45% of participants rated the traditional TTO exercise as ‘difficult’ (n = 15; no participants rated it ‘very difficult’). In contrast, 61% of participants rated the combined TTO exercise as ‘difficult’ or ‘very difficult’ (n = 20; of which n = 5 rated the exercise as ‘very difficult’).


Summary of Findings

In this study, we piloted an approach to explore whether responses to TTO tasks differ when participants are provided with additional contextual information around their need for a carer when living in the health state being valued, as well as the carer’s health state. While participants did rate the combined exercise as more difficult than the traditional exercise, the combined exercise appeared to be feasible to conduct in practice. The mean utilities had face validity in that health state A was preferred to health state B, which was preferred to health state C (irrespective of the exercise). Mean utilities for each health state in the traditional exercise were higher than those observed in the EQ-5D-5L crosswalk value set for the UK [32].

Mean differences in utilities between the two exercises were generally larger in magnitude than many of the estimated minimally important differences for EQ-5D-5L [33, 34], suggesting that differences of this magnitude could be meaningful. However, there was inconsistency in the differences in valuations between exercises (different directions), which meant that there was no strong evidence for a consistent carer effect at a group level. Nonetheless, it cannot be concluded that participants were valuing health states very similarly between the two exercises, as illustrated by the heterogeneous individual response trajectories in Fig. 2. There was also a considerable split between the proportion of individuals who considered health states B and C to be better or worse than dead in both exercises, which added to the noise in the data.

It was expected that many participants would provide lower values in the combined exercise due to a concern for the carer, i.e. the quality-of-life altruism effect observed by Krol et al. [16]. However, a longevity altruism effect may also have occurred with some participants, and some participants may also have been comforted by the confirmation that they would have a carer, both of which could work in the opposite direction. It was also found that the well-being of the carer in the combined exercise was a concern for many participants (n = 15; 45%). The carer's health state in the combined exercise is likely to have differed to the actual health state of the individual(s) that participants imagined for the carer role (Table 5). These differences in health status could have exacerbated either of the altruism effects, affecting valuations in either direction. Furthermore, TTO is known to be subject to learning effects, which may have had a greater impact on the results of the combined exercise relative to the traditional exercise, as the former was always conducted after the latter [35]. This could also have affected results in either direction.

This broad mixture of potential impacts may go some way in explaining the variation in our results, and highlights the challenges associated with identifying the impact of providing contextual information on valuations. Thus, while the task appeared feasible to conduct in practice, identifying the specific impact of the provision of additional contextual information using this approach might not be feasible without substantial adjustments being made to the approach (see Sect. 4.3).

Study Limitations

This pilot study had several limitations. First, the study sample size was small and the study was underpowered for the effect size observed. With only 33 participants, the study would be powered to pick up mean differences of a magnitude of around 0.2, which is larger than the observed mean differences of around 0.07. Second, convenience samples are susceptible to selection effects (explaining our relatively homogenous sample) and our results are likely to have differed if we had recruited a representative sample of the general population. Third, we aimed to follow good practice when conducting the study [19, 25], with a particular focus on EQ-VT protocol [28]. However, we did not employ the EQ-VT digital aid and instead used physical props, given the flexibility required for the combined exercise. We therefore did not benefit from the interactive features of the computer-based tool and the feedback module that has been shown to improve data quality [36]. However, we sought to maximise data quality by providing all interviewers with the same initial training, which was followed up with one-on-one coaching sessions, and by including a warm-up exercise in the interviews. Fourth, while it may be typical to estimate mean utilities using Tobit models that control for sociodemographic characteristics, our analysis focused on direct comparisons of mean differences using paired t-tests. We felt this was justified by our limited sample size, the lack of variation in sociodemographic characteristics, and the need for a panel set up to incorporate observations from both exercises; however, our analysis may be limited as a result. Fifth, while the carer was defined as an informal carer at the beginning of the combined exercise, five participants stated that they considered professional (formal) carers, suggesting that not all instructions were followed precisely by all participants. Finally, another limitation was that the interviews were not audio recorded and it was therefore not possible to obtain full transcripts. Transcripts would have provided richer data for the qualitative analysis.

Future Research

While our pilot study has shown that providing contextual information in health state valuation exercises is feasible, it has also illustrated the potential difficulties in understanding the differences in quantitative results. Nonetheless, further research in this area would be beneficial and this pilot study may provide useful insights for such studies.

Future studies of this nature may benefit from taking a simpler approach to the present study, potentially controlling for competing effects such as mentioning the need for a carer but not specifying their health state (or simplifying the health state provided). Furthermore, while this pilot study did directly refer to carers as opposed to loved ones more generally, it may have been beneficial to have explicitly instructed participants to consider the information in their valuations as this may have been more impactful than simply providing the information, as the results of the study by Krol et al. [16] suggest.

Many participants (n = 22; 67%) stated that their responses would have differed had the carer been in full health, suggesting that this information was relevant to the valuations being made. However, informal feedback from interviewers suggested that some participants struggled to take on board multiple EQ-5D-5L health states at the same time, suggesting that the inclusion of this information increased the difficulty of the exercise. An alternative approach that would enable the impact of the carer health states to be explored further would be to have two combined exercises, where only the carer’s existence is described in the first exercise and the carer’s health state is introduced in the second exercise. Furthermore, the carer's health state could be described using a carer-specific instrument such as CarerQol to reduce the focus on EQ-5D-5L and provide a more relevant description [37].

Future studies may also benefit from collecting more data and recording interviews for transcripts. Directly asking participants to reflect on whether the information about needing a carer is comforting or concerning overall would be useful. It would also be helpful to identify participants who may be unwilling to trade life-years due to longevity altruism, as this has an opposite effect to quality-of-life altruism, but occurs as a result of a similar thought process (i.e. considering the impact on others) [16].

Additionally, future studies may also benefit from directly asking participants whether they believe that the implications of their responses are consistent with their beliefs. In some interviews, interviewers noticed contradictions in the responses provided by participants. Using alternative methods that have a more deliberative focus, or implementing direct validity tests into the study design, could be more informative and result in better-quality preference data [38, 39]. It may also be worth considering that a fully qualitative approach could be superior to a quantitative or mixed-methods approach in this area, given the wide range of factors that may influence valuations and the lower sample size requirements that these require.


This study sought to explore whether health state valuations differed when contextual information about the need for a carer (and the carer’s health state) was provided, and to gauge the extent to which participants implicitly consider the impact of health states on others. While the mean differences in utilities were not statistically significant between exercises, our results show a high level of individual variation in responses, in different directions, suggesting that the inclusion of such information may have an impact. Further research is required to better understand this variation and the results of this pilot study could be useful in informing the design of such studies in future.


  1. Schulz R, Beach SR. Caregiving as a risk factor for mortality: the Caregiver Health Effects Study. JAMA. 1999;282:2215–9.

    CAS  Article  PubMed  Google Scholar 

  2. Pennington B, Wong R. Modelling carer Health-related Quality of Life in NICE Technology Appraisals and Highly Specialised Technologies. Sheffield: Decision Support Unit; 2019

  3. Pennington BM. Inclusion of carer health-related quality of life in National Institute for Health and Care Excellence appraisals. Value Health. 2020;23(10):1349–57.

    Article  PubMed  Google Scholar 

  4. Krol M, Papenburg J, van Exel J. Does including informal care in economic evaluations matter? A systematic review of inclusion and impact of informal care in cost-effectiveness studies. Pharmacoeconomics. 2015;33:123–35.

    Article  PubMed  Google Scholar 

  5. Dixon P, Round J. Caring for carers: positive and normative challenges for future research on carer spillover effects in economic evaluation. Value Health. 2019;22:549–54.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Devlin NJ, Brooks R. EQ-5D and the EuroQol Group: past, present and future. Appl Health Econ Health Policy. 2017;15:127–37.

    Article  Google Scholar 

  7. National Institute for Health and Care Excellence. Guide to the methods of technology appraisal. London: NICE; 2013.

    Google Scholar 

  8. McLoughlin C, Goranitis I, Al-Janabi H. Validity and responsiveness of preference-based quality-of-life measures in informal carers: a comparison of 5 measures across 4 conditions. Value Health. 2020;23:782–90.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Devlin NJ. ‘Preference-based measure’ is misleading—can we agree on something better? The Academic Health Economists’ Blog. 2020.

  10. Dolan P, Gudex C, Kind P, Williams A. The time trade-off method: results from a general population study. Health Econ. 1996;5:141–54.;2-N.

    CAS  Article  PubMed  Google Scholar 

  11. Torrance GW. Measurement of health state utilities for economic appraisal. J Health Econ. 1986;5:1–30.

    CAS  Article  PubMed  Google Scholar 

  12. Baker R, Robinson A. Responses to standard gambles: are preferences‘well constructed’? Health Econ. 2004;13:37–48.

    Article  PubMed  Google Scholar 

  13. van der Pol M, Shiell A. Extrinsic goals and time tradeoff. Med Decis Mak. 2007;27:406–13.

    Article  Google Scholar 

  14. Robinson A, Dolan P, Williams A. Valuing health status using VAS and TTO: what lies behind the numbers? Soc Sci Med. 1997;45:1289–97.

    CAS  Article  PubMed  Google Scholar 

  15. Karimi M, Brazier J, Paisley S. How do individuals value health states? A qualitative investigation. Soc Sci Med. 2017;172:80–8.

    CAS  Article  PubMed  Google Scholar 

  16. Krol M, Attema AE, van Exel J, Brouwer W. Altruistic preferences in time tradeoff: consideration of effects on others in health state valuations. Med Decis Mak. 2016;36:187–98.

    Article  Google Scholar 

  17. Tilling C, Krol M, Tsuchiya A, Brazier J, van Exel J, Brouwer W. Does the EQ-5D reflect lost earnings? Pharmacoeconomics. 2012;30:47–61.

    Article  PubMed  Google Scholar 

  18. Shiroiwa T, Fukuda T, Ikeda S, Shimozuma K. QALY and productivity loss: empirical evidence for “double counting.” Value Health. 2013;16:581–7.

    Article  PubMed  Google Scholar 

  19. Attema AE, Edelaar-Peeters Y, Versteegh MM, Stolk EA. Time trade-off: one methodology, different methods. Eur J Health Econ. 2013;14:53–64.

    Article  PubMed Central  Google Scholar 

  20. Al-Janabi H, Manca A, Coast J. Predicting carer health effects for use in economic evaluation. PLoS ONE. 2017;12:e0184886.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  21. Al-Janabi H, Van Exel J, Brouwer W, Trotter C, Glennie L, Hannigan L, et al. Measuring health spillovers for economic evaluation: a case study in meningitis. Health Econ. 2016;25:1529–44.

    Article  PubMed  Google Scholar 

  22. Devlin N, Parkin D, Janssen B. Methods for analysing and reporting EQ-5D data. Cham: Springer International Publishing; 2020.

    Book  Google Scholar 

  23. Joshi N, Hensen M, Patel S, Xu W, Lasch K, Stolk E. Health state utilities for acute myeloid leukaemia: a time trade-off study. Pharmacoeconomics. 2019;37:85–92.

    Article  PubMed  Google Scholar 

  24. Shen J, Breckons M, Vale L, Pickard R, for the OPEN trial investigators. Using time trade-off methods to elicit short-term utilities associated with treatments for bulbar urethral stricture. PharmacoEconomics Open. 2019;3:551–8.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Shen J, Hill S, Mott DJ, Breckons M, Vale L, Pickard R. Conducting a time trade-off study alongside a clinical trial: a case study and recommendations. PharmacoEcon Open. 2019;3(1):5–20.

    Article  PubMed  Google Scholar 

  26. Janssen BMF, Oppe M, Versteegh MM, Stolk EA. Introducing the composite time trade-off: a test of feasibility and face validity. Eur J Health Econ. 2013;14:5–13.

    Article  PubMed Central  Google Scholar 

  27. Ramos-Goñi JM, Oppe M, Stolk E, Shah K, Kreimeier S, Rivero-Arias O, et al. International valuation protocol for the EQ-5D-Y-3L. Pharmacoeconomics. 2020;38(7):653–63.

    Article  PubMed  Google Scholar 

  28. Oppe M, Rand-Hendriksen K, Shah K, Ramos-Goñi JM, Luo N. EuroQol protocols for time trade-off valuation of health outcomes. Pharmacoeconomics. 2016;34:993–1004.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Hsieh H-F, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–88.

    Article  PubMed  Google Scholar 

  30. Shah KK, Mulhern B, Longworth L, Janssen MF. Views of the UK general public on important aspects of health not captured by EQ-5D. Patient. 2017;10(6):701–9.

    Article  PubMed  Google Scholar 

  31. Sterne JAC. Sifting the evidence—what’s wrong with significance tests? Another comment on the role of statistical methods. BMJ. 2001;322:226–31.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  32. van Hout B, Janssen MF, Feng Y-S, Kohlmann T, Busschbach J, Golicki D, et al. Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Health. 2012;15:708–15.

    Article  PubMed  Google Scholar 

  33. McClure NS, Sayah FA, Xie F, Luo N, Johnson JA. Instrument-defined estimates of the minimally important difference for EQ-5D-5L Index Scores. Value Health. 2017;20:644–50.

    Article  PubMed  Google Scholar 

  34. McClure NS, Sayah FA, Ohinmaa A, Johnson JA. Minimally important difference of the EQ-5D-5L index score in adults with type 2 diabetes. Value Health. 2018;21:1090–7.

    Article  PubMed  Google Scholar 

  35. Augestad LA, Rand-Hendriksen K, Kristiansen IS, Stavem K. Learning effects in time trade-off based valuation of EQ-5D health states. Value Health. 2012;15:340–5.

    Article  PubMed  Google Scholar 

  36. Wong ELY, Ramos-Goñi JM, Cheung AWL, Wong AYK, Rivero-Arias O. Assessing the use of a feedback module to model EQ-5D-5L health states values in Hong Kong. Patient. 2018;11:235–47.

    Article  PubMed  Google Scholar 

  37. Brouwer WBF, van Exel NJA, van Gorp B, Redekop WK. The CarerQol instrument: a new instrument to measure care-related quality of life of informal caregivers for use in economic evaluations. Qual Life Res. 2006;15:1005–21.

    CAS  Article  PubMed  Google Scholar 

  38. Lipman SA, Brouwer WBF, Attema AE. What is it going to be, TTO or SG? A direct test of the validity of health state valuation. Health Econ. 2020;29(11):1475–81.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Devlin NJ, Shah KK, Mulhern BJ, Pantiri K, van Hout B. A new method for valuing health: directly eliciting personal utility functions. Eur J Health Econ. 2019;20:257–70.

    Article  PubMed  Google Scholar 

Download references


The authors are grateful to the 33 participants of this study and to the four interviewers for providing their time to conduct the interviews: Stephen Bradley, Davneet Judge, Tom Loughran and Ajay Patel. The authors are also grateful for the contributions of several other individuals at various stages throughout the project: Prof. Nancy Devlin (for input at the study design stage), Dr Maria Errea (for contributing to interviewer training), Gemma Frisby (for acting as research co-ordinator), Dr Chen Li (for contributing to data analysis) and Dr Evan Campbell (for comments on an earlier draft of the manuscript).

Author information



Corresponding author

Correspondence to David J. Mott.

Ethics declarations


This study was sponsored by Roche Products Ltd. The Office of Health Economics received funding from Roche Products Ltd to act as an academic collaborator on the project, providing advice on the study design, delivering interviewer training, reviewing analyses, and contributing to dissemination activities.

Conflict of interest

David Mott is an employee of the Office of Health Economics, a registered charity, and an Independent Research Organisation, which receives funding from a variety of private and public sector sources. Koonal Shah, a former employee of the Office of Health Economics during the design of this study, is now an employee of PHMR Ltd, which receives consulting income from a variety of sources, including pharmaceutical companies. Jennifer Rowell and Iain Leslie were previously employees of Roche Products Ltd, during the design of this study. Nicolas Scheuer holds shares in Roche and Novartis through employee shares schemes; Jennifer Rowell holds shares in Roche through an employee share scheme; and Iain Leslie previously held shares in Roche through an employee share scheme, all of which were sold at the end of his employment.

Availability of data and material

The de-identified datasets and code generated during the current study will be considered for sharing by the corresponding author upon reasonable request.

Ethics approval

A favourable ethical opinion for the study was obtained from an independent reviewer working under the auspices of the Association of Research Managers and Administrators in November 2019.

Informed consent

All participants provided informed, written consent prior to participating in a TTO interview, which included consent for participation in the interview, as well as consent for de-identified data to be used in future publications.

Author contributions

The study was conceived by Iain Leslie and Jennifer Rowell, and all authors contributed to the study design. Material preparation, data collection, and analysis were led by Nicolas Scheuer and David Mott, with assistance from the individuals listed in the acknowledgements section. The first draft of the manuscript was written by David Mott, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 302 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mott, D.J., Leslie, I., Shah, K. et al. Impact of Including Carer Information in Time Trade-Off Tasks: Results from a Pilot Study. PharmacoEconomics Open 5, 665–675 (2021).

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: