FormalPara Key Points for Decision Makers

This paper examines the extent to which respondents do not attend to certain dimensions of the EQ-5D-5L when valuing health states using the EuroQol Valuation Technology standardised valuation protocol.

The results suggest that respondents are less likely to consider physical dimensions of the EQ-5D-5L (self-care and usual activities) when valuing health states.

The impact on utility scores obtained through discrete choice experiment data depends on the underlying reasons for respondents not considering certain dimensions of the EQ-5D-5L.

1 Introduction

Health-related quality of life (HRQoL) has been defined as a multidimensional concept that focuses on the impact health status has on quality of life [1]. The Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID) lists 414 survey instruments that collect generic measures of HRQoL [2]. Using a generic instrument, it is possible to assign preference weights to health states in a manner that can inform cost-utility analyses [3, 4]. EQ-5D instruments are the most commonly used generic, preference-based measures of HRQoL [3], informing the allocation of significant amounts of healthcare resources [5]. The three-level (EQ-5D-3L) and five-level (EQ-5D-5L) versions describe HRQoL across five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression [5,6,7]. A key feature of EQ-5D instruments is the availability of value sets to weight health state descriptions to reflect societal preferences [8]. One advantage of the EQ-5D-5L compared with the EQ-5D-3L is the establishment of a standardised protocol, enabling the construction of EQ-5D-5L value sets using a common methodology [9,10,11].

The standardised protocol—the EuroQol Valuation Technology (EQ-VT) protocol—recommends that EQ-5D-5L preference data be collected using the composite time trade-off (cTTO) valuation technique, supplemented by a discrete choice experiment (DCE). DCEs were not used in the estimation of many of the EQ-5D-3L value sets, but DCEs can overcome some of the limitations of other methods such as time trade-off (TTO) and standard gamble [12]. For example, indicating a preference when faced with a discrete set of options (e.g. “would you prefer to be in health state X or health state Y?”) is likely to be less cognitively challenging than questions that require a respondent to consider trading life-years (as in the TTO) or their attitude to health risks (as in the SG). The inclusion of the DCE method within the standardised protocol for collection of EQ-5D-5L health state valuations followed a significant period of piloting and testing of different approaches [10].

However, the use of DCE data is not unproblematic. Previous studies in other contexts have shown that respondents may only consider a subset of attributes (i.e. the health dimensions in the context of the EQ-5D instruments) that define the choices made. This phenomenon, known more generally as attribute non-attendance (ANA), is well-recognised in the DCE literature [13,14,15,16,17,18,19]. Several studies have shown that not accounting for ANA can lead to biased coefficients [18]. The reasons for ANA can be important in determining the size of the bias in welfare estimates. ANA can occur because respondents ignore attributes to simplify choice tasks as a form of decision heuristic, or ANA can occur simply because respondents do not consider attributes in their choices when they do not care about them [18, 19]. In the case of decision heuristics, studies have shown how the design of the DCE or choice task complexity affects ANA within a DCE [20, 21]. For example, in a recent study that was based on an EQ-5D-5L experiment, Jonker et al. [21] examined the impact of colour coding and using designs with attribute level balance as methods to reduce the rates of ANA. The findings from previous research suggest that design elements or choice task complexity can affect ANA, although, as noted by Hensher [20], relevancy is also important.

The EQ-VT protocol requires data to be collected using computer-assisted face-to-face interviews with trained interviewers. The interview process includes an instruction to remind respondents to read aloud the health states and consider all dimensions when completing the DCE tasks. Previously, Hole et al. [18] explored ANA using a subset of data that informed the Australian national value set for the EQ-5D-3L [22], although that study was not based on the EQ-VT protocol. In this study, our objective is to examine ANA for the EQ-5D-5L that was collected in accordance with the EQ-VT protocol and to explore the potential impact of ANA on value sets obtained from the DCE data.

2 Methods

2.1 Study Design and Data Collection

The paper uses data described by Hobbins et al. [23, 24] and Barry et al. [25]; details of the study design and data collection process can be found there. In brief, the research design and data collection process followed version 2 of the EQ-5D-5L valuation protocol developed by the EuroQol Group and deployed the EQ‐VT, which was developed specifically for valuing EQ-5D-5L health states using computer‐assisted personal interviews [23]. For the DCE part of the study, respondents were randomly assigned to complete one block of seven DCE pairs from 28 blocks of a possible 196 pairs of health states. An example of a DCE choice card used in the survey is presented in Fig. 1.

Figure 1
figure 1

source: EuroQol Valuation Technology [EQ-VT] protocol) [26]

Example of the discrete choice experiment task (

Full size image

A team of trained interviewers collected the data and followed the quality control process developed by the EuroQol Group.

2.2 Econometric Analysis

The standard framework for the analysis of DCE data is the random utility model (RUM) [27]. The model assumes that respondents will choose the alternative that provides the highest level of utility based on the attribute levels of that alternative. Attributes vary across choice cards such that the effect of each attribute level on the probability of selecting an alternative can be estimated. The RUM model assumes that utility is composed of an observable component \(\beta n^{\prime}x_{nit}\) and an unobserved random component \(\varepsilon_{nit}\) (Eq. 1):

$$U_{nit } = \beta n^{\prime}x_{nit } + \varepsilon_{nit}$$
(1)

where \(\beta n\) represents a vector of coefficients used to describe preferences for the x attributes. Within the conditional logit (CL) framework, the probability that respondent \(n\) chooses alternative \(i\) in choice occasion \(t\) is given by Eq. 2:

$$P_{nit } |\beta_{n} = \frac{{{\text{exp}}\left( {\beta_{n}^{^{\prime}} x_{nit} } \right)}}{{\mathop \sum \nolimits_{j = 1}^{J} {\text{exp}}\left( {\beta_{n}^{^{\prime}} x_{njt} } \right)}}$$
(2)

The CL is commonly used on DCE data for health preference studies [28] and the model gives a preference weight to each dimension for each respondent regardless of whether some respondents may not have attended a particular dimension. To accommodate ANA, we use an equality constrained latent class (ECLC) modelling approach as proposed by Scarpa et al. [14] and subsequently applied in different studies to model ANA (for example [15, 18, 19, 29,30,31]). In this model, \(\beta n\) is assumed to follow a discrete distribution and belong to one class \(c\) of \(C\) classes. Thus, the probability conditional on class \(c\) that respondent \(n\) prefers alternative \(i\) over alternative \(j\) is shown in Eq. 3

$$P_{nit } |\beta_{c} = \frac{{{\text{exp}}\left( {\beta_{c}^{^{\prime}} x_{nit} } \right)}}{{\mathop \sum \nolimits_{j = 1}^{J} {\text{exp}}\left( {\beta_{c}^{^{\prime}} x_{njt} } \right)}}$$
(3)

The unconditional choice probability is shown in Eq. 4:

$$P_{nit } = \mathop \sum \limits_{c = 1}^{C} \frac{{{\text{exp}}\left( {\theta_{c} } \right)}}{{\mathop \sum \nolimits_{c = 1}^{C} {\text{exp}}\left( {\theta_{C} } \right)}} P_{nit } |\beta_{c} = \frac{{{\text{exp}}\left( {\beta_{c}^{^{\prime}} x_{nit} } \right)}}{{\mathop \sum \nolimits_{j = 1}^{J} {\text{exp}}\left( {\beta_{c}^{^{\prime}} x_{njt} } \right)}}$$
(4)

The probability of belonging to a particular class \(c\) is given by the first expression, \(\theta_{c}\) are the \(c - 1\) class membership model parameters and the \(C{\text{th}}\) parameter vector is normalised to zero to ensure model identification [30, 31]. In the ECLC model, if there is non-attendance to an attribute, then its parameter is set to zero, so the attribute does not affect choice probabilities. Second, the attribute parameters that are not set to zero are constrained to be the same across all classes [30], so the model accounts for differences in how respondents process attributes (or ANA) [14].

2.3 Analytical Strategy

There are 32 (25) possible combinations of ANA processing strategies (including where respondents consider all dimensions) based on the five dimensions of health within the EQ-5D-5L. For example, some respondents may consider only mobility, self-care and usual activities (i.e. three of five) when making their choices, others may consider pain/discomfort only (one of five), while some may consider all dimensions.

In this paper, rather than estimating a model with all combinations of ANA processing strategies adopted, which could lead to problems of over-identification, we begin our analysis with a smaller set of classes, remove non-significant classes, and then re-estimate the model. First, we specify a class where we assume that all coefficients across the five dimensions affect choices (i.e. assume full attendance to all dimensions). We also add five additional classes estimated simultaneously. Each of these classes sets the dummy coefficients on one dimension of the EQ-5D-5L to zero and estimates the probability associated with each class. For example, one class would represent a processing strategy where respondents considered all dimensions of the EQ-5D-5L except for mobility. We do this for each of the five dimensions. Once we have estimated this six-class specification, we inspect the class membership probabilities. For any classes that were not significant or had low class membership probabilities (< 5% of respondents estimated to belong to that class), we removed that class. We then re-estimated a new model and included the significant classes from step one alongside new classes in which we assumed that respondents did not consider pairs of dimensions. We followed similar steps and examined three-dimension ANA (where three dimensions were not attended to) and four-dimension ANA (where respondents only considered one of the EQ-5D-5L dimensions). We removed classes that were not significant or had low membership probability. We also examined the Akaike information criterion (AIC) and Bayesian information criterion (BIC) to examine the overall impact on model performance with the addition or removal of ANA classes. As a sensitivity analysis, we used the endogenous attribute attendance (EAA) model, developed by Hole [16], to explore ANA using a modelling approach that is not based on stepwise identification of ANA.

As a final part of the analysis, we consider the impact that ANA has on the utility scores for health states using estimates obtained from the DCE data only. We generate utility scores for all 3125 possible combinations of responses to the EQ-5D-5L based on the DCE data, and we anchored the coefficient values using the value set estimated by Hobbins et al. [23]. Anchoring may not be ideal for rescaling as it could bring the values between the models closer together, but we do not have another mechanism to rescale the coefficients in the present study. To determine the impact of ANA on value set estimates from the ECLC model, the interpretation of the underlying cause of ANA is important [18, 19]. As noted by Hole et al. [18] and Heidenreich et al. [19], if we assume that ANA is because of preferences (i.e. respondents did not consider dimensions that they did not care about), then we weight the ECLC model coefficients by the probability of attendance (multiply the coefficients by the probability of attendance) and then we anchor the coefficients. In this regard, the utility weights reflect the population mean utility weights, and, under this interpretation of ANA, we can treat it as a form of preference heterogeneity [18]. On the other hand, if we assume that respondents have ignored dimensions to simplify the choice tasks, even though they would otherwise value them, then we cannot retrieve utility weights for them. Therefore, the utility weights can only be estimated for respondents who considered the dimensions, and they are computed directly from the anchored coefficients from the model, similar to the approach in the CL model. In this case, for respondents who have ignored the attributes due to heuristics, we do not have information to compute utility weights for them.

3 Results

In total, interviewers collected data from 1160 respondents. Detailed discussion of the sample is presented by Hobbins et al. [23, 24]. Table 1 presents the results from the CL model, the ECLC model and the EAA model for comparison purposes. In these models, the dependent variable is a binary variable, with zero representing the chosen alternative; therefore, the parameters on the worse health states are positive.

Table 1 Results from the CL and ECLC models
Full size table

Following our analytical strategy, we chose a six-class ECLC model. The overall model that we chose had class membership probabilities of at least 5% per class and it performed only slightly worse on the BIC criterion than class models with a large number of classes, which had many non-significant classes with small membership probabilities.

In our reported ECLC model, approximately 40% of respondents are estimated to consider all dimensions of the EQ-5D-5L. We summed the different probabilities of ANA across classes to estimate the overall level of ANA across the five dimensions for the ECLC model, and we also present the estimates of ANA for the five dimensions estimated from the EAA model in Table 2.

Table 2 Estimated ANA probabilities for the ECLC and EAA models
Full size table

In the ECLC model, pain/discomfort was the most attended dimension, with 6% of respondents estimated to not consider this dimension. Less than one-quarter of respondents did not attend to the anxiety/depression dimension. Based on the ECLC model, respondents were more likely to ignore the more physical functioning dimensions of the EQ-5D-5L, with 48%, 48% and 38% of respondents not considering the self-care, usual activities and mobility dimensions, respectively. For the EAA model, we found that there were some differences in the rates of ANA for two of the dimensions. The largest difference was for pain/discomfort, which was estimated to be ignored by 25% of respondents in the EAA model. While the estimated rate of ANA was somewhat lower for the mobility dimension under the EAA model (25% under the EAA model vs. 38% under the ECLC model), while similar rates of ANA for the other dimensions were found between the two models.

3.1 Impact of Attribute Non-attendance on Health State Valuations Obtained from Discrete Choice Experiment Data

Based on the decision-heuristics explanation of ANA, there is a large impact of ANA on estimates, as shown in Table 3. For example, under this interpretation, approximately 21% of health states have a utility difference of 0.10 or greater in computed utility scores between the CL and ECLC models. Under the assumption that ANA is because respondents did not consider dimensions because they do not care about them, the impact of ANA on value sets is small, as shown in column 3. Given that we do not know the underlying reasons for ANA, it is prudent to examine the sensitivity of estimated value sets to both potential explanations for ANA. Moreover, different respondents may exhibit ANA processing strategies because of either heuristics or because they do not care about certain dimensions.

Table 3 Variation in health state valuations across models based on DCE data
Full size table

4 Discussion

This paper is the first to consider ANA within the context of the EQ-VT standardised protocol for valuing EQ-5D-5L health states. The paper shows that approximately 40% of respondents considered all the dimensions of the EQ-5D-5L, meaning that 60% employed some form of an ANA processing strategy based on the results from the ECLC model. Other studies that have used DCEs in health have reported higher rates of ANA using similar modelling approaches to those reported here. For example, Lagarde [15] and Heidenreich et al. [19] have reported rates of full attendance at < 3% and 5%, respectively. We acknowledge though that these studies estimated more latent classes than we did in this study, which might explain the differences. It could also be the case that the acquired experience embedded in the EQ-VT protocol, including the use of trained interviewers, face-to face data collection and the inclusion of quality control checks, all contributed to mitigating ANA somewhat in this study.

One finding from the study is that the underlying reasons for ANA are important for health state valuations. While we did not have data on reasons for ANA within this study, we can examine qualitative statements that were collected following completion of the valuation exercises to help shed some light on how respondents’ viewed the choices. Some respondents specifically stated that they ignored certain dimensions (e.g. mobility, self-care, usual activities) as they regard pain/discomfort and/or anxiety/depression as key determinants of health. For example, one respondent noted they “Only focused on pain and depression and disregarded others entirely. Sees them as incomparable in terms of management and life quality”. Another respondent mentioned that “I struggled to decide between pain and anxiousness, they were my deciding factors”. Other respondents mentioned being able to cope with degradations in other dimensions. For example, as noted by one respondent “Having the background knowledge through my professional life has coloured my view this way. I am aware that I would overcome any physical difficulties/obstructions rather than suffer a chronic mental health issue. I also became aware that I felt guilty during this survey for not wanting to live the life of someone with a mental health issue”. Other respondents explicitly mentioned that the poor quality of mental health services in Ireland was a factor in their choices, e.g. “For me often it was about balancing my mental health state with the level of pain. Even with physical problems, with a good mental state you could push through. It’s hard to make the decisions because you can only really relate to your own experience or maybe what you have seen people close to you go through, it’s tough to imagine things you have no experience with. Health services, at the moment are better equipped to deal with physical problems but we have to look more into services and care for mental health issues”. Other respondents mentioned how experience of seeing others suffer from pain or anxiety had affected their choices. As one respondent noted, “I am a healthcare professional so having experience of co-morbidities, it is never straightforward but living in some of the states without any pain relief or help with anxiety or depression then there would be very little quality of life”.

While the qualitative statements are interesting and suggest that some of the ANA might be due to preferences, it would be valuable to explicity examine reasons for ANA within the context of health state valuations. Currently, the qualitative statements that are collected as part of the EQ-VT are not designed specifically to capture reasons for ANA. If heuristic decision making is common in health state valuations, studies have shown some relatively straightforward design mechanisms that might help mitigate this (see, for example, Jonker et al. [21]), which may be useful to consider in further studies that are designed to estimate national value sets.

A limitation of this paper is that we have considered the impact of ANA in isolation. It is likely that some of the estimated ANA is confounded with other types of heterogeneity. For example, some respondents may have considered some dimensions but given them less weight, and this could lead to these respondents being mischaracterised as non-attenders [32]. Therefore, the results could represent the upper bounds in terms of ANA in this data set, similar to Hole et al. [18]. Moreover, there may be other differences between respondents, such as differences in the degree of scale heterogeneity, which could impact findings. We were interested in exploring potential patterns of ANA and therefore we estimated an ECLC model. However, using a stepwise approach to determine the inclusion of latent classes can impact the estimated coefficients, p-values and standard errors, and it is unlikely to capture the full patterns of ANA within the data. As a result, we included the EAA specification as it does not rely on stepwise identification of classes. However, other modelling approaches could be estimated; for example, models that accommodate preference heterogeneity and ANA simultaneously that do not rely on stepwise identification of ANA [e.g. 21, 31]. Moreover, our study did not include any indicators within the survey instrument to determine whether ANA was the result of preferences or decision heuristics. Including such an indicator within the survey would have helped to determine more realistically the impact of ANA on health state valuations.

5 Conclusion

While many studies have found that ANA is present in DCE datasets, the impacts of ANA on results tends to be study-specific (i.e. some studies have found large impacts while others have not). Given this observation, and the estimation of many national valuation sets for the EQ-5D-5L (and other preference-based instruments), ongoing consideration of this issue could offer further reassurance on the impact of ANA in other valuation studies and whether ANA is the result of decision heuristics or preferences.