Background

Health-related quality of life (HRQOL) has become an increasingly important outcome of health care and public health interventions. HRQOL can be assessed using different instruments. The EQ-5D-3 L instrument is a standardized HRQOL questionnaire, which was developed by the EuroQol Group to provide a simple, generic measure of health for clinical and economic appraisal [1]. The EQ-5D-3 L questionnaire is available in more than 160 translated versions [2], and is the most commonly used instrument worldwide for assessment of utilities for use in cost-utility analyses to appraise health care interventions [3]. The EQ-5D is widely used in clinical trials, observational studies, and other health surveys [4, 5].

To aid in the interpretation of psychological tests, scores are often compared with a normative group, i.e. a group of test-takers that are representative of the population for whom the test is intended. Norms provide a frame of reference to interpret an individual’s scores relative to scores of others, whether the norms are national or local [6]. National population norms for the EQ-5D by socio-demographic characteristics are available from many countries [7], including the UK [8], Sweden [9], Finland [10], Denmark [11], Italy [12], Poland [13], Switzerland [14], United States [15, 16], China [17], Japan [18], Sri Lanka [19] and more [7]. The population norms present distributions of the five item scores, EQ VAS scores, or EQ-5D Index values that are based on different value sets and methods, e.g. the European VAS value set algorithms, or country-specific VAS or time trade-off (TTO)-derived values [7].

Most national and regional EQ-5D population surveys have used face-to-face interviews or computer-assisted personal interviews [7]; however some studies used postal surveys [20,21,22,23] or computer-assisted telephone interviews [12, 24, 25]. Some studies also pool data from different postal surveys, for example to establish population norms for the EQ-5D [11]. For another commonly used health status measure, the SF-36 version 1, data from telephone and face-to-face interviews were combined to represent US general population norms [26]. However, we are not aware of population norms from web-based surveys for the EQ-5D.

Several Norwegian studies have used the EQ-5D-3 L to examine HRQOL in many diseased populations, including people with epilepsy [27], COPD [28], stroke [29, 30], diabetes [31], rheumatoid arthritis [32], hip fractures [33,34,35], HIV/AIDS [36], brain tumors [37], and multiple myeloma [38]. To date there are no available population norms for this instrument in Norway.

This study aimed to provide population norms for the EQ-5D-3 L according to age, and sex, using nationally representative samples of Norwegians aged 18 years and above. The study compared results from a postal survey and a web-based data collection. We hypothesized that there would be no difference between the postal and web-based surveys after adjustment for age, sex and education.

Methods

Samples and surveys

The study consisted of two study arms, 1) a postal survey and 2) a panel study with electronic data collection. We sent invitations in both surveys to samples that were aimed to be reasonably representative of the Norwegian general population with regard to sex, age and education (details below). The postal survey was carried out from May 10 to June 18, 2010, and the electronic survey from June 7 to June 22, 2010. In both surveys, we sent one reminder.

The authors prepared a common core questionnaire that was adapted for postal and electronic data collection by TNS Gallup, a market research company. The postal and panel versions of the questionnaire were similar, with the exception of how the respondents responded to the EQ VAS, a valuation of health conditions on a 0–100 point scale. The postal version had a vertical visual analog scale, while the electronic version used a solution with a horizontal slider. When moving the slider from left to right, the respondent decided on a value between 0 and 100.

The study was approved by the local privacy ombudsman for research at Akershus University Hospital on 27 April 2010, noting that the project was considered to be anonymous information, and presentation for the Regional Committee for Medical Health Research Ethics was not considered necessary.

Postal survey

The list of names and associated registered addresses was selected from the National Registry of the Norwegian Tax Administration (Folkeregisteret). A company, EDB Businesspartner, was after application granted access to the registry and selected a random sample of 5000 persons from the total Norwegian population aged 19–100 including age, postal address, and marital status. TNS Gallup was responsible for data collection, including carrying out the postal survey using the name/address list. After the survey the responses were anonymized before the data file was given to the authors. Hence, we only had access to age, sex, marital status and county of habitation from the registry. The postal questionnaire had an enclosed return envelope. Half of the participants in addition received a lottery ticket with a value of NOK 25 (≈ 2.5 Euros) as an incentive to respond.

Panel survey with electronic data collection

The sample for the electronic data collection was drawn from TNS Gallup’s access panel, “GallupPanelet”. This panel included about 60,000 willing survey participants. They had been recruited through various previous telephone (landline and mobile phone) and postal surveys conducted by TNS Gallup. The panel was maintained continuously by recruitment, updating of background variables and automatic withdrawal of participants that had completed a certain number of surveys. Because of the size and recruitment method it was possible to select nationally representative samples > 18 years of age. Such samples were selected in two steps; first strata according to age, gender, address were created; then the final sample was selected within the strata.

The response rate in a panel survey cannot easily be compared with the response rate in a postal survey. The panel participants have been recruited in advance, and the survey is closed when the required number of respondents has been achieved.

The electronic part of the data collection used an electronic HTML-based questionnaire. We invited potential participants through an e-mail invitation, with a link to the questionnaire. The respondents to the panel were compensated by an incentive, where they received points according to a normed length of the questionnaire in minutes (1 point = 1 min). The points could be collected as a gift card or donated to a charity. The value of 1 point was NOK 1 (0.1 Euro) at the time of the survey.

The EQ-5D instrument

The EQ-5D instrument is a self-completed questionnaire, which can be completed on paper, tablet or on the web. It consists of the EQ-5D descriptive system that measures HRQOL today on five dimensions (mobility, self-care, usual activities, pain/discomfort, and anxiety/depression). This study used the 3 L version, EQ-5D-3 L, having three scoring levels in each dimension. In addition, the instrument contains a 20 cm vertical visual analog scale, the EQ VAS, rating “your own health state today” on a 0 (worst imaginable) to 100 (best imaginable) scale.

Based on the responses to the descriptive system, the scores can be aggregated to a single index value, the EQ-5D Index, which can be used in the estimations of Quality-Adjusted Life Years (QALYs). The EQ-5D Index is generated on the basis of an algorithm reflecting the preferences of general populations, or tariffs. Several country-specific tariffs exist. As no tariff is available for Norwegians, this study used the oldest and most commonly used tariff, the UK tariff [39], for generating EQ-5D Index values in the Norwegian population.

Statistical analysis

We compared descriptive statistics for the respondents in the postal and web surveys using the t-test or Fisher’s exact test and the distributions of scores on the EQ-5D single items using the Fisher’s exact test.

To adjust for age, sex, and education, we attempted to use ordered logistic regression analysis with the ordinal EQ-5D dimension scores as the dependent variable. Because few respondents scored at level 3 in the Mobility and Self-care dimensions, we combined the “Some problems/Confined to bed” for the Mobility dimension, and the “Some problems/Unable to wash or dress” for the Self-care dimension. For these two dimensions, the survey results were compared using multivariable logistic regression analysis. In the three remaining dimensions, Usual activities, Pain/Discomfort, and Anxiety/Depression, we used ordered logistic regression analysis. The proportional odds assumption was checked using the Brant test and was met. In these models, we used three co-variates (age (continuous), sex and education (< 10 years, 10–13 years, > 13 years).

We compared EQ VAS, EQ-5D Index scores and EQ VAS-based values using the t-test for unadjusted scores, and used multivariable linear regression analysis to adjust for age, sex and education.

We used Stata version 14.2 (StataCorp, College Station, TX) or R version 3.2.0 (The R Foundation for Statistical Computing) for statistical analyses. A significance level of 5% was chosen, using two-sided tests.

Results

Response and representativeness

In the postal survey, 101 questionnaires among 5000 were returned because of unknown address. In total 1276 individuals responded to the questionnaire, of whom 1131 (22.6% of the gross sample, 23.1% of the net sample) completed both the EQ-5D and the EQ-VAS and were used in the further analysis.

The panel participants were recruited in advance, and the survey was closed when the targeted number of respondents had been reached. Of a total of 1936 panel participants that received the invitation, 1192 opened the survey, and 1003 (51.8%) completed it, and 977 had valid responses to the EQ-5D items. The response rate in the electronic survey cannot easily be compared with the response rate in a postal survey.

The final samples had a good spread according to sex, age and educational attainment. The respondents had a mean (SD) age of 51.8 (17.4) and 50.7 (14.7) years in the postal (n = 1131) and web (n = 977) survey, respectively. Age ranged from 19 to 97 years in the postal and 19 to 86 years in the web survey. The age distribution between the respondents in the two surveys differed (p < 0.001), with more respondents in the 61–70 years group (26%) and fewer > 70 years (6%) in the web survey than in the postal one (18% and 15%, respectively). Also the distribution of educational attainment differed (p < 0.001), as the web survey had a larger proportion of respondents with basic education and lower proportion with higher education (Table 1). The proportion of female respondents was similar in both surveys (p = 0.56).

Table 1 Descriptive statistics for respondents. Number (%) unless stated otherwise
Full size table

Comparison of scores between surveys

The distribution of scores in the Mobility, Self-care, Usual activities, and Anxiety/Depression dimensions did not differ between the two surveys, both for unadjusted scores and in the odds of being in a higher score category after adjustment for age, sex and education (Table 2). There was, however, a difference in the scores in the Pain/Discomfort dimension, with a larger proportion reporting no pain (level 1) in the postal survey than in the electronic one.

Table 2 Distribution of item scores according to sample, number (%), and comparison of scores between modes of survey
Full size table

EQ-VAS scores were similar in the postal and web surveys in the various strata according to age, sex and education, except for lower unadjusted and adjusted score for web respondents aged 41–50 years and for those with higher education (≥14 years) than for postal respondents (Table 3).

Table 3 EQ-VAS scores (0–100 scale) according to sample in different strata and comparisons of scores between modes of survey using linear regression analysis
Full size table

Comparison of UK EQ-5D tariff scores between the surveys showed a similar pattern, with lower unadjusted and adjusted scores in the web survey for respondents aged 41–50 years and those with higher education (≥14 years) than for postal respondents (Table 4).

Table 4 EQ-5D index scores using the UK tariff according to sample in different strata, and comparison between modes of survey using linear regression analysis
Full size table

Normative scores

The distributions of the pooled EQ-5D item scores from the two surveys are presented as norm sets according to age and sex, indicating a lower proportion level 1 (lowest) score with increasing age, except for in the Anxiety/Depression dimension (Table 5). Normative scores are shown separately for men (Table 6) and women (Table 7).

Table 5 Distribution of EQ-5D item scores. Men and women according to age. Pooled postal and web surveys. Number (%)
Full size table
Table 6 Distribution of EQ-5D item scores. Men only, according to age. Pooled postal and web surveys. Number (%)
Full size table
Table 7 Distribution of EQ-5D item scores. Women only, according to age. Pooled postal and web surveys. Number (%)
Full size table

The UK EQ-5D tariff scores for Norway are shown separately for the postal, web and pooled surveys (Table 8), indicating a consistent pattern across age and gender, with two exceptions: 1) a slight dip in scores for males aged 51–60 years in both surveys, when compared with similarly aged women and with younger and older males, and 2) a dip in scores for females aged 41–50 in the web survey.

Table 8 UK EQ-5D index scores for Norway, by panel, sex, and age. Mean (SD)
Full size table

Discussion

This is the first study to examine the Norwegian population norms for the EQ-5D instrument. The major finding was that the distribution of scores for the EQ-5D descriptive system and its derived utility scores were rather similar in a postal survey and a panel survey with electronic data collection. Therefore, we think it is justified to pool the values in a combined norm set.

In the two surveys, and hence in the pooled sample, there was a deterioration of health status with increasing age, with some minor exceptions, as shown in previous studies and in a catalogue of population norms for the EQ-5D index and the EQ-VAS [7]. We noted a dip in EQ-5D index scores in the 51–60 years group among males, or an increase in the 61–70 years group. Another study noted a similar dip in EQ-5D index and other HRQoL scores at ages 55–64, or a slight hump in HRQoL in the 65–74 age group, compared to a linear, downward trend associated with age group; however, this was not considered a systematic effect [16]. The difference between genders in the present study was consistent across the EQ-5D index and EQ-VAS, in line with previous studies [7].

As far as we know, no previous collection of EQ-5D-3 L population norms has been collected using a web sample. Therefore, the difference in results between web-based and postal administrations is unknown. In principle, using a web panel could be problematic for two reasons: First, the mode of administration could impact on how people respond. Second, a web panel is by definition self-selected, and the question is whether the sample is representative of the general population.

Addressing the first question, a review of papers studying the agreement between electronic and paper administration of patient-reported outcome measures, including EQ-5D-3 L, found that the two modes of administration had a high level of equivalence [40]. The review was mostly based on studies of diseased populations, however, there is no obvious reason why the general population should have a larger difference in responses based on mode of administration than patients. Increasingly, patients who participate in studies fill out the EQ-5D electronically, and the new standard for performing valuation studies involves the use of computer-assisted interviews [41]. In that respect, electronically collected population norms have ecological validity since the mode of administration is the same.

McHorney et al. noted that in a randomized trial on the effects of data collection method on the summary scores of a different health status instrument, the SF-36, respondents to telephone interviews tended to score higher on the MCS than those in a postal survey, although there was no effect on the PCS [42]. Furthermore, telephone-administered US general population norms for the SF-36v2 in 2005–2006 [43] were higher than for norms collected by mail administration in 1998 [44] which was attributed to the effect of telephone administration.

Response rates to epidemiological surveys in general [45] and in the Nordic countries have declined over the last decades [46]. The consequences of the use of web surveys have been debated and challenges raised [47]. Often, results obtained with electronic versions are generally similar to paper versions, in terms of outcomes and psychometric properties, although the response rates to internet surveys tend to be lower and the respondents less representative than in mail surveys [48, 49]. However, established offline norms for established offline tests may still not be appropriate for use with online versions, as evidenced from psychometric tests in different settings [50]. Moreover, Buchanan states that when comparing online scores with normative data, “normative data gathered online must be used” [50]. This may apply for HRQoL as well.

Self-selection bias is a general problem when conducting surveys; people who agree to participate in surveys are systematically different from people who decline [51]. However, one could assume that web panels consist of individuals who are the most inclined to participate in studies. To counter this effect, survey companies are able to put extra effort into recruiting individuals with typical non-responder characteristics to their panels, for instance individuals from the youngest and oldest age groups. As an additional step in studies where representativeness is important, the companies may over-sample groups which have low response rates.

The pooled postal and web sample in this study was comparable to the Norwegian general population regarding gender and education, although the proportion aged 19–40 years was somewhat lower and the proportions 41–60 and > 60 years of age were somewhat higher, than in the general population in 2010 [52]. We did not weight the sample to better approximate the general population. Some studies of HRQoL have pointed at various method for adjusting web-based panel results to better approximate the scores of the general population, such as post-stratification adjustment [53,54,55]. One of these studies reported web-based norms for two HRQoL instruments, the AQoL-6D and AQoL-8D, using post-stratification weights to address the effects of self-selection in the web-based survey [55].

Internet coverage is important when assessing the potential bias in using a web panel, since in some cases only selected parts of a population – the wealthy and highly educated, has had access to the relevant technology. In Norway, using computers connected to the internet is free in public libraries. Further, there is a high coverage rate of internet in Norway, as in 2010 90% of the Norwegian households had access to the internet, and 97% in 2016 [56].

Recently, the Norwegian Medicines Agency has published draft guidelines for pharmaco-economic analyses [57]. The agency recommends using the EQ-5D for estimating utilities, and suggests using Swedish population norms [9, 23] as a reference, because Norwegian norms are unavailable. The means and SDs for the population values in the various age strata in the present study were very similar to the Swedish norms collected in 1998 [9]. The norms in both the Swedish and the present study were based on the same EQ-5D-3 L tariff from the UK [39], although the Swedish study assigned a value of 0 to states rated as worse than death [9], in contrast to the present study and the original UK tariff. There are also other important methodological differences. For example, the Swedish norms were based on regional data only (from Stockholm County) from a postal survey, and it had three postal reminders and a telephone follow-up of non-respondents resulting in a higher response rate than in the present study. Furthermore, the Swedish study presented data separately for those > 80 years of age [9].

Some limitations of this study should be noted. We have discussed some of the short-comings and problems with a web-based panel, where it is difficult to evaluate the non-response and representativeness as traditionally done in postal surveys. The sample size was sufficient to provide normative values according to sex and age groups in 10-year intervals, although the number of respondents above > 70 years was too small for splitting this into several age groups. This may be important, as health status might be expected to deteriorate rapidly with increasing age > 70 years.

Conclusion

Our observations of equivalence between general population normative values collected in a web panel aimed to be representative of the general population and those from a postal survey, in which the survey was sent to random individuals drawn from the Population Register of Statistics Norway, supports the use of web samples when collecting EQ-5D population norms. Because of the similarity of the scores, we have pooled the scores to a larger data set, which may be used as general population norms for the EQ-5D-3 L in Norway. This may be useful for example in assessment of health care interventions or in pharmaco-economic analyses.