Respondents
A sample of respondents was drawn from members of a commercial panel. Only persons between 18 and 65 years of age were approached to participate in the online experiment. Stratification to represent the Dutch population was based on gender, education, and age. Respondents were not given a financial reward for participating.
Health-state selection and description
Health states were based on the Dutch version of the five-level EQ-5D (EQ-5D-5L) [12]. This instrument consists of five domains of health: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. The instrument has five answer categories for each domain, generating 3,125 (55) health states. Out of the total of 3,125 possible health states, 100 were selected in light of a previously developed D-optimal design [13].
Study design
All respondents performed a combination of tasks. First, they filled out a background questionnaire. They also indicated how they perceived their own health on the EQ-5D-5L instrument and the EQ-5D visual analog scale. Scores on the latter ranged from 0 to 100, where 0 stood for the worst imaginable health and 100 the best imaginable. Then the respondents had to choose which of two EQ-5D-5L health states they considered best in a paired comparison task. Upon completing these preliminary tasks, the respondents were randomized over five different specifications of TTO: lead-time TTO with a duration of 15 years and of 20 years; lag-time TTO with a duration of 15 years and of 20 years; and classic TTO with a duration of 10 years. Within these five specifications, respondents were randomized over ten blocks containing ten EQ-5D-5L health states, and each state was presented in random order. The study ended with a short feasibility questionnaire.
The TTO tasks
In classic TTO, health-state values are elicited by asking respondents if they would prefer living x years in a period of full health to living t years in impaired health where x < t. If respondents accept living a shorter period x in full health, they are essentially willing to trade length of life for quality of life. The health-state value is then given by x/t, at the point of indifference. When the respondents would rather trade off all healthy life years than have to live in a particular health state for period t, they indicate that this health state is worse than dead (WTD), at least when the duration of that health state is equal to period t. Respondents then enter a different task to measure their negative preference values (since x < 0). In this WTD task, they are asked to choose between immediate dead and a life of duration t, with x years in full health preceded by t−x years in the imperfect health state. The value for the health state following this WTD task is generally −x/(t−x). In lead (or lag) time TTO, they were also asked if they would prefer living x years in full health compared to living t years in impaired health preceded (or followed) by l years in full health. An indifferent point was estimated by repeating this question for different values of x. The value of the health state is then given by (x−l)/t, where x is the estimated indifference value. When x < l, the formula results in a negative value, implying that these are WTD health states.
The TTO tasks were preceded by an animated instructional video. It explained how to trade off life years by giving an example with a hypothetical EQ-5D state, whereby an animated figure of a ‘doctor’ pointed out the various elements of the task. The video was designed to highlight the characteristics of the different TTO tasks. Thus, the examples shown in each animation preceding the real TTO task were identical in characteristics and layout to the real TTO task that followed, with the exception that the health state that was presented was not used in the study.
The classic TTO is a two-part task. The visual design and the health-state value equations for health states better than dead are different from those for WTD health states. The other four TTO tasks have a uniform visual representation and health state value equations for better than dead and WTD valuations. In all tasks, respondents are asked to choose between a fixed period in Life A and a variable period t in Life B. The value of x depends on the respondents’ previous choice for either Life A or B and follows the fixed iteration procedure described below.
Iteration procedure
The first two ‘steps’ of the fixed iteration procedure were similar for all five TTO tasks. At the first iteration, respondents were asked to choose between two scenarios: Life A, which contained the health state and, depending on the task, a lead-time or lag-time in full health, and Life B, which was set at the maximum of all years in full health (health-state value = 1, or x = 10, 15, or 20, depending on the total time frame). At the second iteration, the health-state value of Life B was 0 (or x = 0 for the classic TTO and x = 10 for the other variants). If respondents preferred Life A at value = 0, they would indicate that the health state is WTD. If they preferred Life B, they would indicate that the health state is better than dead. After this ‘sorting question,’ the iteration procedure continued with a choice between Life B and Life A where the value of B was set at x for value = 0.5 or −0.5. Conditional on choosing Life A or B, the remaining iterations represented value increments or decrements of 0.1 or 0.05 with the corresponding values of x in Life B.
Health-state value equations
The equations applied for the lead-time TTO in a 20-year time frame are (without discounting):
$$ 10U_{FH} + 10U_{{HS_{i} }} = xU_{FH} $$
(1)
where U
FH
is the value (utility) of full health, U
HSi
the value of the health state i, and x the number of years in full health at which the respondent indicated being indifferent in the TTO task. Solving for U
HSi
gives:
$$ U_{{HS_{i} }} = \frac{x - 10}{10} $$
(2)
For a respondent who considers x = 13 years in full health equal to 10 years in full health followed by 10 years in health state i, the value for i is: U
HSi
= (13−10)/10 = 0.3. In the same vein, the equation for lag-time TTO is:
$$ 10U_{{HS_{i} }} + 10U_{FH} = xU_{FH} $$
(3)
Equation 3 can also be solved for U
HSi
, which again results in Eq. 2. The most relevant details of the TTO specifications included in this study are described in the “Appendix” to enable easy comparison with other studies performed with a TTO checklist [14].
Analysis
All respondents who completed the online exercise were included in the analyses. To check for consistency in findings, the analyses were rerun in a smaller sample without those respondents who: (1) indicated on the feasibility questionnaire that they did not understand the task; (2) did not differentiate among any of the ten health states; or (3) had used only three or fewer iterations for all health states.
Comparison of health-state values
Mean lead-time TTO and lag-time TTO values were compared for all 100 health states. The different minimum health-state values set for the TTO methods distort comparisons of the mean values between tasks. For example, solving the equations for t = 0 (trading in all life years) results in U = −2 for a ratio of lead-time to disease time of 2:1 and U = −1 for a ratio of 1:1. Therefore, comparisons of the mean are only made for tasks with similar attainable health-state values. Convergence of lead-time TTO and lag-time TTO with classic TTO was measured in terms of the mean absolute difference (MAD) to get a feel for the comparability of values despite the different ranges of health-state values.
The relative importance of the domains of EQ-5D in the different specifications of TTO is compared through random effects regression analysis to take account of the panel structure of the data (multiple TTO observations per respondent). Although the sizes of the coefficients are not directly comparable because of different ranges of the dependent variable (the TTO values), the relative importance of the domains within each regression model can still be compared. Independent variables in the regression model were the EQ-5D health domains, applied as continuous variables.
The online mode of administration of the TTO is still in an experimental stage. Also, the health-state values generated by the different tasks cannot be compared to a non-experimental EQ-5D-5L tariff, as the valuation protocol of the EQ-5D-5L was still under development at the time of this study. To get an indication of the convergent validity of the values produced in the online exercise, these values were compared to the estimated EQ-5D-5L values derived from a mapping function [15]. These estimates reveal which health-state value is expected for an EQ-5D-5L health state on the grounds of previous valuations for the EQ-5D-3L applied in face-to-face TTO.
Task engagement and response characteristics
Agreement among respondents in the different TTO tasks was ascertained with Levene’s test and Brown and Forsythe tests. It was assumed that differences in valuations between respondents, regardless of the cause, would result in greater variance and thus a less precise health-state value estimate. Although larger standard deviations may reflect preference heterogeneity rather than poorer task engagement, a valuation method that is identical in all respects but the onset of the health state (i.e., before or after a period of full health) is arguably preferable if there is more agreement among respondents. Variances for classic TTO (with transformed negative values) were only compared to the TTO tasks with a 20-year time frame, as TTO values for these two lie on the same −1 to 1 scale. Accordingly, the variances were not compared to values from the TTO tasks with a 15-year time frame (with a lead time to disease time ratio of 2:1), which lie on a −2 to 1 scale and thus logically have larger variances. Standard deviations, which lend themselves to a more intuitive interpretation than variances, were plotted for lead-time TTO and lag-time TTO. Other indicators of task engagement were used as well: whether the respondents were willing to trade off any time at all (non-traders); how many iterations the respondents used before reaching their point of indifference; how many respondents ‘used up’ all tradable time; and how many did not differentiate between health states.
Feasibility
Differences between tasks were compared using four items of a feasibility questionnaire presented after the TTO task. Respondents were asked to indicate their level of agreement with four statements: (1) The instructions that were given made it clear what I needed to do; (2) it was easy to understand the questions I was asked; (3) I found it difficult to decide on the exact point where Life A and B were about the same; (4) I found it easy to tell the difference between the health states I was asked to think about. The answer categories ranged from 1 (completely agree) to 5 (completely disagree). The mode, median, and percentiles of the answers on these questions were compared.
Since health-state values have been shown to be affected by the number of health states valued by a respondent, we repeated our analysis using only the first five valued health states [16]. We tested for significance of order effects by regressing the sequence of a health state on the number of iterations using ordinary least squares (OLS) regression, as proposed by Augestad et al. [16]. All statistical analyses were run in STATA 11.