Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study

Abstract

Purpose

The aim of this study was to assess the measurement properties of the 5-level classification system of the EQ-5D (5L), in comparison with the 3-level EQ-5D (3L).

Methods

Participants (n = 3,919) from six countries, including eight patient groups with chronic conditions (cardiovascular disease, respiratory disease, depression, diabetes, liver disease, personality disorders, arthritis, and stroke) and a student cohort, completed the 3L and 5L and, for most participants, also dimension-specific rating scales. The 3L and 5L were compared in terms of feasibility (missing values), redistribution properties, ceiling, discriminatory power, convergent validity, and known-groups validity.

Results

Missing values were on average 0.8 % for 5L and 1.3 % for 3L. In total, 2.9 % of responses were inconsistent between 5L and 3L. Redistribution from 3L to 5L using EQ dimension-specific rating scales as reference was validated for all 35 3L–5L-level combinations. For 5L, 683 unique health states were observed versus 124 for 3L. The ceiling was reduced from 20.2 % (3L) to 16.0 % (5L). Absolute discriminatory power (Shannon index) improved considerably with 5L (mean 1.87 for 5L versus 1.24 for 3L), and relative discriminatory power (Shannon Evenness index) improved slightly (mean 0.81 for 5L versus 0.78 for 3L). Convergent validity with WHO-5 was demonstrated and improved slightly with 5L. Known-groups validity was confirmed for both 5L and 3L.

Conclusions

The EQ-5D-5L appears to be a valid extension of the 3-level system which improves upon the measurement properties, reducing the ceiling while improving discriminatory power and establishing convergent and known-groups validity.

Introduction

As a generic preference-based measure of health, the EQ-5D has many applications that aid decision making in health care [1]. The standard format of the EQ-5D descriptive classification system developed by the EuroQoL Group consists of five dimensions of health, each with three levels of problems (EQ-5D-3L, hereafter “3L”). Over the past twenty years, value sets for the 3L health classification system have been developed for many countries around the world [2].

There is an extensive body of literature to support the validity and reliability of the 3L descriptive system, the EQ-VAS, and the 3L index values in many conditions and populations [3–8]. However, its restricted ability to discriminate small to moderate differences in health status has been questioned widely [9–12]. Moreover, several studies reported a ceiling effect of the 3L in both general population and patient settings [13–18].

The EuroQol Group has recently introduced a 5-level EQ-5D (EQ-5D-5L, hereafter “5L”), which expands the range of responses in each dimension from three to five levels [19]. Preliminary studies indicated that prototype 5L versions improved upon the properties of the 3L in terms of reduced ceiling effects, increased reliability, and improved ability to discriminate between different levels of health [20–22].

A Korean study has shown good measurement properties for the 5L in cancer patients [23]. To our knowledge, there has been no validation of other language versions of EQ-5D-5L, nor has there been assessment of measurement properties in other patient groups or a combination of patients groups. The goal of this study was to assess the measurement properties of the 5L, in comparison with the 3L, across a wide range of patient groups. The specific aims were to evaluate and compare the properties of 3L and 5L in terms of feasibility (missing values), consistent redistribution of responses from 3L to 5L, ceiling, discriminatory power (Shannon indices), convergent validity, and known-groups validity.

Methods

Data

This study aimed at assessing measurement properties for 3L and 5L in eight broad patient groups. A student cohort was added in order to investigate how both instruments perform in a healthy population sample. Respondents completed both the 3L and 5L in six countries: Denmark, England, Italy, the Netherlands, Poland, and Scotland. Data collection in Denmark was conducted through the endocrinology, rheumatology, and orthopedic departments of a regional university hospital. Data collection in England was organized through a specialist patient recruitment agency and aimed at patients with prespecified conditions. In Italy the cohort of liver disease patients completed the questionnaires locally at two hospitals (Bergamo and Naples). Data collection in the Netherlands was conducted at a specialist center for personality disorders and at a local hospital for the kidney dialysis patients. In Poland, the student cohort was recruited at the Medical University of Warsaw in Poland, and the stroke cohort was recruited through the Neurological Clinic in Warsaw. Data collection in Scotland took place through a specialist patient recruitment agency, with patients completing the questionnaires at primary care centers. Paper and pencil versions of the questionnaires were used in all countries except in England where data collection took place online. Data collection took place between August 2009 and September 2010. The 5L was administered first, followed by the EQ-5D visual analogue scale (EQ-VAS) and a number of demographic questions, then the 3L, and finally a set of five dimension-specific rating scales. All respondents scored 5L first, as a previous study showed a tendency to avoid the in-between levels 2 and 4 of 5L when responding to the 3L first [20]. Data collection was undertaken with informed consent and according to the ethical guidelines for health research in each country.

Measures

The 3L version of the EQ-5D is the initial version that has been used in many clinical trials and methodological studies published in the peer-reviewed literature [1]. It is a brief self-reported generic measure of current health that consists of five dimensions (Mobility, Self-Care, Usual Activities, Pain/Discomfort, and Anxiety/Depression), each with three levels of functioning (no problems, some problems, and unable to/extreme problems). This health state classification describes 243 unique health states that are often reported as vectors ranging from 11111 (full health) to 33333 (worst health). Societal value sets have been derived from population-based valuation studies around the world that, when applied to the health state vectors, result in preference-based index values that typically range from states worse than dead (<0), to 1 (full health), anchoring dead at 0. In addition, the EQ-5D includes an EQ-VAS where own health “today” is rated on a scale from 0 (worst imaginable health) to 100 (best imaginable health).

In developing the 5L, the five-dimensional structure of the 3L was retained, but the descriptors within each dimension were adapted to a 5-level system based on qualitative and quantitative studies conducted by the EuroQol group [19]. The labels for 5L followed the format no problems, slight problems, moderate problems, severe problems, and unable to/extreme problems for all dimensions. For Mobility, the description of “confined to bed” was changed to “unable to walk about.” Additionally, for Usual Activities, the word “performing” was changed to “doing” (English for UK version). The official EQ-5D-3L and EQ-5D-5L language versions for each country were used.

For the purposes of the current study, respondents also rated their own health “today” on five dimension-specific rating scales, one for each of the EQ-5D dimensions. Each scale consisted of a horizontal hash-marked line (from 0 to 100) with corresponding numbers (0, 10, 20, …, 100). The descriptive anchors at each end of the scales were the same anchors as used in the 3L and 5L, that is, no problems and unable to/extreme problems.

Convergent validity was assessed by comparing the 3L and 5L dimensions to the WHO-5 Well Being questionnaire. The WHO-5 captures well-being and was developed from the World Health Organization-Ten Well-Being Index [24, 25]. It was conceptualized as a unidimensional measure that contains five positively worded items: “I have felt cheerful and in good spirits”; “I have felt calm and relaxed”; “I have felt active and vigorous”; “I woke up feeling fresh and rested”; and “My daily life has been filled with things that interest me,” all operationalized using a six-point Likert scale ranging from 0 (not present) to 5 (constantly present). A sum-score can be calculated as a summary measure.

Analysis

Feasibility was assessed by calculating the number of missing values for 3L and 5L. The ceiling of the EQ-5D was defined as the proportion of respondents scoring no problems on any of the five dimensions, that is, the proportion of respondents scoring 11111. Under the assumption that the majority of patients should have at least some problem on at least one of the EQ-5D dimensions, we expect the ceiling to be lower for 5L compared to 3L. An absolute reduction when going from 3L to 5L was calculated, but since the ceiling was very small in some patient groups, a percentage reduction was also calculated: (ceiling_3L − ceiling_5L)/ceiling_3L.

Redistribution properties of the 3L to 5L extension

Redistribution properties and (in)consistency of responses were evaluated using criteria established in previous studies [20, 21]. An inconsistent response was defined as a 3L response followed by a 5L response that was at least two levels away. The redistribution properties of the consistent response pairs were described as proportions of the 3L–5L response pairs within each 3L response level (3L-1, 3L-2, and 3L-3) and corresponding dimension-specific rating scale values. For valid redistribution, dimension-specific rating scale values should be increasing when going from the “healthiest” response pair (3L-1 paired with 5L-1) to the most extreme response pair (3L-3 paired with 5L-5).

Discriminatory power

The Shannon index and the Shannon Evenness index were used to assess discriminatory power. Originating from the field of information theory, the Shannon index has been widely used in ecological studies as a measure of biodiversity and in molecular biology as a measure of the information content of DNA molecules [26–28]. Previous research showed Shannon’s methodology to be useful in assessing discriminatory power in health state classifications [20, 21, 23, 29, 30]. In the present study, we estimated discriminatory power for each dimension separately. The Shannon index is defined as:

$$ H^{\prime} = - \sum\limits_{i = 1}^{L} {p_{i} \log_{2} } p_{i} $$

where H′ represents the absolute amount of informativity captured, L is the number of levels, and p _i  = n _i /N, the proportion of observations in the ith level (i = 1, …, L), where n _i is the observed number of scores (responses) in level i and N is the total sample size [31]. The higher the index H′ is, the more information is captured by the system. In the case of a uniform (rectangular) distribution (i.e., p _i  = p* for all i), the optimal amount of information is captured and H′ has reached its maximum (H′_max) which equals log₂ L. If the number of levels (L) is increased, H′_max increases accordingly, but H′ will only increase if the newly added levels are actually used. Shannon Evenness index (J′) exclusively reflects the evenness (rectangularity) of a distribution, regardless of the number of levels. Shannon Evenness index (J′) is defined as: J′ = H′/H′_max. The Shannon indices are calculated by dimension and also by instrument as a whole, treating each health state vector as a unique category.

The Shannon indices are purely descriptive measures of the discriminatory power of a classification system and have no relation to the content, meaning, or clinical relevance of what the instrument aims to measure. Both the Shannon index and the Shannon Evenness index are needed to make a useful interpretation of the discriminatory power of a measurement scale. Consider any 3L and 5L dimension: Clearly, the 5L has more discriminatory potential. However, if the extra levels are not used, the H′ value will be the same in both dimensions. Therefore, the Shannon Evenness index J′, which will be lower, is needed to express the loss in potential of the 5-level dimension. Conversely, when both the 3L and 5L show rectangular distributions, the J′ value will be the same. In this case, H′ is needed to express the better discriminatory performance of the 5L. We expected H′ to increase and J′ to marginally decrease at most.

Convergent validity

Convergent validity between the 3L and 5L dimensions and the WHO-5 items was assessed using Spearman rank order coefficients (Spearman’s rho), including a comparison with the WHO-5 sum-score. We hypothesized correlations to be highest for WHO-5 items with Anxiety/Depression. Convergence of 3L and 5L with dimension-specific rating scales was also assessed.

Known-groups validity

Known-groups validity was tested for all 3L and 5L dimensions in regard to age, education, and smoking status. Tests for age-groups (18–24, 35–44, 45–54, 55–64, 65–74, and 75+) and education were performed using Spearman rank order coefficients, and smoking status (never smoked, ex-smoker, and current smoker) was assessed with the Kruskall–Wallis H statistic. Education was included in three substudies (England, Denmark, and Scotland) and was recoded into three levels (1 = primary/lower secondary; 2 = secondary/vocational; 3 = higher/college). In regard to known-groups validity, we expected a lower reported health status for respondents with increasing age, lower education, and respondents who smoke or have smoked. In order to take possible clustering effects into account, we applied a set of statistical techniques developed for nonparametric statistics for clustered data, with country as cluster variable [32, 33].

The study data were analyzed centrally using PASW version 18.0.0 and R version 2.15.2.

Results

In total, 3,919 respondents completed both the 3L and 5L (Table 1). The overall cohort was 52 % female and had a mean age of 51.9 (standard deviation (SD) 20). A mean (SD) EQ-VAS score of 64 (23) was observed, ranging from 41 (30) for Parkinson’s disease to 79 (16) for the student sample. For 5L, 683 unique health states were observed (22 % of the total number of theoretically possible health states) versus 124 for 3L (51 % of the total).

Table 1 Characteristics and descriptive results of study sample by country and patient group

Respondents were classified into nine different subgroups that included cardiovascular disease (n = 251), COPD/asthma (n = 342), depression (n = 250), diabetes (n = 284), liver disease (n = 645), personality disorders (n = 384), rheumatoid arthritis/arthritis (n = 372), stroke (n = 614), and students (n = 443). Less prevalent conditions listed in Table 1 were collapsed into an “other conditions” category (n = 334). The average number of unique health states by subgroup was 49 for 3L ranging from 16 (student population) to 73 (stroke patients), and 158 for 5L ranging from 49 (student population) to 280 (stroke cohort).

Missing values ranged from 43 for Mobility (1.1 %) to 57 for Pain/Discomfort (1.5 %) for 3L and from 19 for Mobility (0.5 %) to 37 for Usual Activities (0.9 %) for 5L. Missing values were on average 0.8 % for 5L and 1.3 % for 3L, indicating good feasibility for both instruments.

Cross tabulations of responses to the 3L and 5L, which include all data, showed that participants reported a wide range of level of health within each of the EQ-5D dimensions (Table 2). The areas shaded gray in Table 2 show the inconsistent responses. The number of inconsistencies was highest in Pain/Discomfort (n = 130; 3.4 %) and lowest in Mobility (n = 82; 2.1 %). The average proportion of inconsistencies by dimension was 2.9 %.

Table 2 Cross tabulation for EQ-5D-3L and EQ-5D-5L dimension scores (inconsistent responses are marked with italicized values)

Table 3 shows 3L and 5L dimension responses for the eight patient groups and the student cohort. Overall, 5L responses show a good spread for most dimensions and patient samples, revealing the benefit of the extra levels in the 5L. The responses in Mobility show the effect of changing the most extreme level from “confined to bed” to “unable to walk about,” as respondents make better use of the 5L scale.

Table 3 Dimension responses for EQ-3D-3L and EQ-3D-5L across eight patient groups and a student cohort

Redistribution from 3L to 5L using the dimension-specific rating scales as reference showed valid results for all 35 3L–5L (consistent) level combinations, as the mean rating scale scores decreased when going from the healthiest subgroup to the most disabled subgroup, regardless of dimension (Table 4). Proportions (% by level) show considerable variation across dimensions. For the 3L–1 subgroups of each dimension, there was always a higher proportion in 5L–1 than in 5L–2. The most skewed relative frequency distribution was in Self-Care (97/3) and the least in Pain/Discomfort (86/14). The 3L–2 subgroups showed variable proportions per dimension; the most evenly spread proportion was in Mobility (37/38/25) and the most unevenly spread in Anxiety/Depression (50/41/9). The 5L–4 scores always corresponded with the lowest proportion for 3L–2. The 3L–3 scores corresponded with the largest proportion in 5L–5 for the first three dimensions, but were associated with more 5L–4 scores in the case of Pain/Discomfort and Anxiety/Depression.

Table 4 Redistribution properties from EQ-5D-3L to EQ-5D-5L: consistent responses

The ceiling by disease subgroup and by country is shown in Table 5. The reduction in ceiling going from 3L to 5L varied considerably over subgroups and countries, ranging from an absolute reduction of 1.1 % for stroke patients to 12.6 % for the student cohort. Percentage reduction ranged from 7.1 % for the Danish population to 49.0 % for the Dutch population. On average, the ceiling was reduced from 20.2 % (3L) to 16.0 % (5L), an absolute reduction of 4.2 % and a percentage reduction of 20.8 %. Overall, the ceiling was reduced the least for the Danish and Italian population samples.

Table 5 Ceiling for EQ-5D-3L and EQ-5D-5L in nine subgroups and six countries

Absolute discriminatory power (Shannon index) showed a substantial gain in information richness by using the 5L classification system for all dimensions and the overall classification system (H′_5L/H′_3L): Mobility (1.89/1.19); Self-Care (1.42/1.05); Usual Activities (2.08/1.39); Pain/Discomfort (2.01/1.28); Anxiety/Depression (1.96/1.30); and overall (4.8/7.2). Relative discriminatory power (Shannon Evenness index) improved slightly for most dimensions and the overall system (J′_5L/J′_3L): Mobility (0.81/0.75); Self-Care (0.61/0.66); Usual Activities (0.89/0.88); Pain/Discomfort (0.87/0.81); Anxiety/Depression (0.85/0.82); and overall (0.62/0.61). On average, absolute discriminatory power improved considerably with 5L (mean 1.87 for 5L versus 1.24 for 3L), and relative discriminatory power improved slightly (mean 0.81 for 5L versus 0.78 for 3L), confirming our hypothesis.

There is evidence of convergent validity of 3L and 5L with the WHO-5 (Table 6). All Spearman rank order coefficients for 3L and 5L comparisons with the five WHO-5 items were significant (p < 0.001). Correlations were highest for Anxiety/Depression, especially with feeling in good spirits (3L = 0.55; 5L = 0.57) and feeling calm and relaxed (3L = 0.61; 5L = 0.61), as expected. High correlations were also found between Mobility, Self-Care, and Usual Activities with feeling active and vigorous (“Energy”), showing correlation coefficients over 0.50 except for 3L Mobility (0.43). The 5L dimensions demonstrated slightly better convergent validity compared with 3L, with the largest difference observed for Mobility. Correlations with the WHO-5 sum-scores were 0.49 for 3L on average (ranging from 0.39 for Mobility to 0.58 for Anxiety/Depression) and 0.53 for 5L on average (ranging from 0.48 for Pain/Discomfort to 0.58 for Anxiety/Depression). Convergence of 3L and 5L with dimension-specific rating scales improved slightly with 5L over 3L (mean Spearman’s rho 0.80 versus 0.77, respectively).

Table 6 Convergent validity: 3L and 5L dimensions with WHO-5 (Spearman rank order coefficients*)

Results for known-groups validity are shown in Table 7 and confirmed our hypotheses. All 3L and 5L correlations with age are significant and in the expected direction, showing increased reported problems for each dimension with increasing age, except for Anxiety/Depression which shows slightly less reported problems with increasing age. Results for education were similar, showing significantly less reported problems with higher education, except for Anxiety/Depression (nonsignificant). Correlations were generally similar for 5L and 3L. Kruskall–Wallis tests showed significant results for all dimensions except 3L Pain/Discomfort. The percentage proportions showed increasing reported problems going from nonsmokers to ex-smokers and smokers as expected. The analyses for clustering showed that for age all comparisons were still significant, although the p values were higher (range 0.004–0.041). For education and smoking cluster analyses resulted in nonsignificant results for all 3L and 5L dimensions. When performing analyses for the separate countries on education, Scotland showed significant results for all 3L and 5L dimensions, England showed significant for all 3L and 5L dimensions except Self-Care and Anxiety/Depression, and Denmark showed nonsignificant results for all 3L and 5L dimensions. In regard to smoking, for all 3L and 5L dimensions England showed significant results and Scotland showed nonsignificant results.

Table 7 Known-groups validity: 3L and 5L with socio-demographic variables^a

Discussion

The aim of this study was to assess the performance of the 5L, in comparison with the 3L, in terms of feasibility (missing values), redistribution properties, ceiling, discriminatory power (Shannon indices), and convergent validity. The 5L performed similar in terms of feasibility, showed increased discriminatory power, slightly improved convergent validity, and similar known-groups validity. Redistribution was confirmed, and the ceiling was reduced with 5L.

The frequency proportions of the redistribution showed varying distributions over the dimensions. As expected, the healthiest subgroup within dimensions (3L-1 paired with 5L-1) always showed the largest proportion, since many (treated) health conditions display no symptoms or problems on a particular dimension no matter how refined the response scale. In all dimensions, the 3L-3 and 5L-4 response pair proportion was large (≥ 20 %). This supports the inclusion of a fourth level at this position, as many respondents opted for “severe problems” on 5L compared to “extreme problems” on 3L–3. The same applies to the response pair 3L-2 and 5L-2, where many respondents opted for “slight problems” on 5L compared to “some/moderate problems” on 3L–3, thus supporting the inclusion of a second level at this position. The response pair 3L-2 and 5L-4 was smaller than expected for some dimensions, that is, 9 % for Anxiety/Depression, 12 % for Pain/Discomfort, and 13 % for Self-Care. It would seem that for these dimensions, “some” or “moderate” problems on 3L are better covered by “slight” or “moderate” problems on 5L, rather than by “severe” problems on 5L.

Due to the lower threshold (i.e., presence of level 5L–2, “slight” problems), we expected a lower ceiling in the 5L version. There was indeed a significant reduction in the ceiling for most patient groups. When the absolute reduction is low but the ceiling is also low, it can be more useful to look at the percentage reduction. This revealed a considerable reduction in the current study (e.g., for COPD/asthma, personality disorder, RA/arthritis, and stroke). For some countries both the absolute and percentage reduction were rather low, however, such as in the Danish and Italian patient samples. It is possible that these are “true” findings: When respondents have no problems on the five dimensions, they will report “no problems” no matter how many levels were added. For the Danish sample, this was supported by the relatively good health status of the participating patients, especially those with diabetes who comprised the main part of the sample. Thus, 39 % of the Danish patients with diabetes reported that the severity of their condition was “mild,” 41 % had no diabetic complications, and 32 % rated their self-perceived health as either “excellent” or “very good.” The Italian sample consisted wholly of liver disease patients with few problems on any dimension regardless of whether the 3L or 5L version was used. Responses for some of the subgroups in this sample, including chronic hepatitis, cirrhosis, and patients who received liver transplantation, might be influenced by effective coping mechanisms to deal with these long-term conditions.

Extending the EQ–5D descriptive system to a five-level version resulted in higher absolute discriminatory power than for the three-level version in all dimensions, as expected. Surprisingly, relative discriminatory power (evenness) did not deteriorate in the 5L but was slightly better than for the 3L version. The high evenness score in all 5L dimensions indicated that the extra levels were used efficiently. Convergent validity with WHO-5 improved with 5L, especially for Mobility, which might be caused by changing the 3L level “confined to bed” to “unable to walk about.” Known-groups validity was confirmed for both 5L and 3L, showing similar results. Cluster analyses had no impact on the analyses for age but brought the results for education and smoking into question. Separate analyses for each country confirmed the hypothesis for all countries again, except for Denmark where education had no impact and for Scotland where smoking had no impact. For Denmark this is likely due to a power issue since the mean level scores all point in the right direction, and the relatively healthy Danish sample shows a rather homogeneous distribution, making it harder to find statistically significant differences. For Scotland possibly the old age of the respondents and the low reported health status might mask the effects of smoking.

The results of this study provide evidence of the validity of the EQ-5D-5L in a range of patient groups across six countries. Not all measurement properties were tested in the current study. The Korean version of the EQ-5D-5L has proved to be reliable in cancer patients [23], but reliability still needs to be determined for other language versions and other patient groups. Furthermore, responsiveness to health changes over time still needs to be assessed. A limitation of the current study is that since 5L was always tested first, there may have been an order effect. The order effect could account for the slightly higher proportion of missing values for the 3L. A further limitation is that since the study was mainly conducted in patient population settings, it was not possible to calculate and apply sampling weights.

Alongside the descriptive classification system, an important aspect of the EQ-5D is the availability of index-based value sets. Valuation studies for the 5L are in progress around the world and are likely to be published in the near future. Until these studies are finalized, index values for 5L based on the 3L value sets are available using a mapping approach, described in detail by van Hout et al. (2012) and on the EuroQol Web site at www.euroqol.org [34].

In conclusion, the EQ-5D-5L is a descriptive system based upon the dimensions of the EQ-5D-3L that demonstrates valid redistribution, reduced ceiling, and improved discriminatory power and convergent validity. Future studies that further examine the properties of the EQ-5D-5L in specific conditions and patient populations, particularly studies comparing the EQ-5D-5L to the EQ-5D-3L, are encouraged.