Introduction

Care provided by family members, friends or other acquaintances, often unpaid, makes up a significant part of the total amount of long-term care provided to individuals in need of help or support due to health problems or frailty as a consequence of ageing. This type of voluntary help is commonly called informal care. Estimates suggest that the prevalence rates of informal care vary from 20 up to 44% in Europe [1]. Informal care has an important impact in economic, social and health terms, particularly in the context of chronic diseases. Especially in countries with tight financial and capacity constraints in their health and social care sectors, and which face ageing populations, informal care is an indispensable part of the total care provided to people in need [2, 3]. It has therefore been advocated to include the effects of interventions on informal caregivers in policy decision-making and (economic) evaluations of health and social care interventions [4, 5]. Such inclusion can have a significant impact on cost-effectiveness of health and social care interventions [6]. Hence, there is an increasing need to measure and value informal care, using validated methods that allow comparisons across populations with different caregiving contexts, help to reduce the methodological heterogeneity and increase transferability of results across countries.

Providing informal care can strongly influence the quality of life of the caregiver as the tasks involved can be physically and emotionally demanding, especially when required over longer periods of time and with high intensity [7, 8]. On the other hand, studies confirmed that caring for a family member or friend can be fulfilling as well [9, 10]. For example, Brouwer et al. [10] showed that, given the illness of a loved one and their need for informal care, the happiness of most caregivers would decrease if they had to hand-over all care tasks to someone else. Hence, informal caregiving can impact several life domains of caregivers, either negatively or positively. The CarerQol questionnaire has been developed to quantify and value the subjective burden of providing informal care, by measuring care-related quality of life of informal caregivers [7, 11, 12]. The CarerQol comprises a descriptive system with seven items (CarerQol-7D) covering both positive and negative impacts of informal care and a visual analogue scale measuring happiness (CarerQol-VAS).

Translated versions of the CarerQol have been developed for several languages already (i.e. English, Dutch, German, Norwegian, Swedish, Italian, Spanish and Portuguese) and these have been successfully used in multi-country surveys [13, 14]. However, language versions for the Central and Eastern European region are currently still lacking. Moreover, notwithstanding exceptions like a study in the USA [15] and one in Australia [16], studies focusing on the validity of the instrument were performed in The Netherlands. Given the substantial differences in the level of happiness, health, cultural context, wealth, as well as health and social care systems across countries [17], investigating the performance of the CarerQol instrument in a sample of informal caregivers from another European region can add valuable information about the cross-cultural validity of this instrument. In addition, previous validations were conducted in distinct samples of caregivers that for instance cared for adult care recipients with a chronic disease or health problems due to ageing [7, 12, 18,19,20], for caregivers of patients in palliative care [16] or for children with special health needs [21, 22]. However, there is less experience with the CarerQol when used in online samples of the general population [19, 20] although there is an increasing trend in using online surveys. The administration mode of the CarerQol may influence response patterns [20], therefore we aimed to test the online version in the Hungarian context.

Most CarerQol validation studies included the EQ-5D-3L, a generic health status measure that involves a 3-level response scale to report problems in five health domains [23]. The recently developed EQ-5D-5L version (with a 5-level response scale) has been proved to have better measurement properties than the EQ-5D-3L in samples from the general population [24,25,26,27]. To our knowledge, the EQ-5D-5L was used in combination with the CarerQol in two studies [13, 28], but these did not focus on the validity of the CarerQol. Hence, investigating the associations between the CarerQol and EQ-5D-5L can provide valuable information about the validity of the CarerQol instrument and for estimating caregiver utilities from care recipient health data when caregiver data are not available. Utilities represent societal preferences for or the value people attach to different health or caregiving states. Using available tariffs, scores on the EQ-5D-5L and the CarerQol can be turned into utility values that can be included in the effect-side of cost–utility analyses of interventions.

The aims of our study were, therefore, to develop the Hungarian version of the CarerQol instrument and assess its construct validity alongside the EQ-5D-5L questionnaire in an online sample of informal caregivers in the general population.

Methods

Data collection

The data presented here were part of a larger three-country survey (Hungary, Poland and Slovenia), details have been published elsewhere [29]. The survey consisted of two main modules. The objective of the first module was to obtain country-specific population tariffs for the CarerQol-7D instrument [30]. The second module (reported in this paper) focused on the subsample of current informal caregivers and their care-related quality of life. A cross-sectional online survey was conducted in November 2018. The web-based questionnaire was distributed by a professional survey company, Big Data Scientist Kft. Respondents were recruited from a large online panel of this survey company. Quotas were applied to ensure the representativeness of the sample (N = 1000) for the general adult population by age, gender, educational level and residency.

The questionnaire and selection of informal caregivers

Data were collected on the social-demographic characteristics of respondents (such as age, gender, education, marital status and current employment status), the household of the respondent (size, monthly net income), the place of residence (settlement type, geographical region) and the health status of the respondent (self-perceived health: excellent/very good/good/fair/poor; and the EQ-5D-5L). To select informal caregivers, first a detailed definition of informal care was provided. Then, respondents were asked about their experience with informal care, either as a recipient or a provider, details have been published elsewhere [29] (Online Resource 1). We followed the same method to select informal caregivers for the analyses [29]: only those respondents were included who selected the “I have been providing care or support to a family member or friend for a long period of time” response in the survey. We did not apply any restrictions regarding the characteristics of the care recipient.

Informal caregivers were asked to complete the CarerQol-7D instrument, and to provide further details on their caregiving situation, including the nature of health problems of the care recipient (mainly mental, mainly physical or both), the health status of the care recipient (using the EQ-5D-5L Proxy questionnaire and the health status scale: excellent/very good/good/fair/poor), and the stability of the health status of the care recipient (no improvement is expected in the future but worsening may occur; improvement is expected in the future, do not know), the relationship to the care recipient (partner, parent, mother/father-in-law, child, brother/sister, grandparent, uncle/aunt, other family member, neighbour, friend), the time allocated to the caregiving tasks (hours per week), for how long the respondent had been providing care to the care recipient (duration of caregiving) and if the care recipient lived in the same household (and if not, the travel time to get there). We also asked respondents to evaluate their overall caregiving experience, i.e. whether caregiving affected the respondent’s life negatively, positively or neither negatively nor positively. The survey questionnaire was pilot tested on five respondents in Hungary.

The instruments

The CarerQol instrument [7] consists of two parts. The first part, the CarerQol-7D descriptive system, includes two positive (fulfilment and support) and five negative (relational problems, mental health problems, problems combining daily activities with care, financial problems and physical health problems) caregiving domains, each with three response levels (none, some, a lot). The second part, the CarerQol-VAS, measures current happiness of caregivers on a horizontal visual analogue scale (VAS) ranging from 0 (completely unhappy) to 10 (completely happy). The answers on the CarerQol-7D descriptive system jointly describe a caregiving situation. For each caregiving situation defined by the descriptive system, a preference-based index score (or utility score) between 0 (representing the worst possible caregiving situation) and 100 (representing the best possible caregiving situation) can be computed. Tariff sets derived from representative samples of the general public are available for several countries [31, 32]. Since Hungarian tariffs are not yet available, CarerQol-7D scores were calculated using the Dutch tariffs [32].

The EQ-5D-5L is a generic health status measure [24]. The questionnaire consists of two parts. First, a descriptive system covering five domains of health: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. Respondents are asked to indicate the level of problems per domain (no problems—1, slight problems—2, moderate problems—3, severe problems—4 and extreme problems/unable to—5) that best describes their current health status. Utility tariffs have been established for the health states described by the instrument for several countries, reflecting the preferences of the general public for the different health states. Due to a lack of country-specific tariffs for Hungary [33], we used the tariffs for The Netherlands in our study (value range: − 0.446 reflecting extreme problems on all domains; 1.0 reflecting no problems in any of the five domains) [34]. Second, the instrument contains a vertical visual analogue scale (EQ VAS), with anchors referring to the worst (0) and best (100) imaginable health state. Respondents are asked to indicate their current health on this scale. We applied the proxy version (EQ-5D-5L Digital Proxy1 Web) to assess the care recipients’ health status, based on caregivers’ responses, as we did not have direct access to care recipients.

The development of the Hungarian version of the CarerQol-7D questionnaire

A Hungarian language version of the CarerQol-7D was developed by independent forward and backward translations and pilot tests. Two independent forward translations from English to Hungarian were performed. The native in-country principal investigator reviewed the forward translations, discussed the differences with the team and the developers of the original CarerQol to produce a reconciled Hungarian version that most accurately represents the concepts within the source version. This version was back translated into English by two independent professional translators (blinded to the original English wording). Translations were checked for accuracy and were compared to the original English language questionnaire, and minor changes of the Hungarian wording were made (e.g. reconciled forward translation of ‘fulfilment’ was ‘megelégedést okoz’; back translations were ‘fulfilment’ and ‘satisfaction’; the final Hungarian version was ‘elégedettséggel tölt el’). After this, cognitive debriefing was conducted in face-to-face interviews with five respondents, before finalising the Hungarian language version.

Ethics

Ethical approval was obtained from the Hungarian Medical Research Council (Nr. 35286-2/2018/EKU). Respondents were informed that participation in the survey was completely voluntary, that their data would remain anonymous and would not be linked to personal information, such as their name or address, and that their data would be used solely for scientific purposes. Respondents needed to provide their informed consent before the start of the survey.

Analysis

The psychometric performance of the CarerQol-7D was evaluated in relation to the health status of both the informal caregiver and the care recipient, as measured with the EQ-5D-5L [24], and caregiving situation characteristics. The construct validation included clinical, convergent and discriminant validity tests. Subgroup comparisons were carried out by non-parametric tests (Mann–Whitney U and Kruskal–Wallis tests). Spearman correlation coefficients were calculated between continuous variables. In all types of analysis, a 5% significance level was applied. The strength of correlation coefficients (r) was assessed as follows [35] r < 0.1: trivial; 0.1 ≤ r < 0.3: small; 0.3 ≤ r < 0.5: moderate; 0.5 ≤ r < 0.7: high; 0.7 ≤ r < 0.9: very high; and 0.9 ≤ r: nearly perfect. All statistical analyses were carried out in SPSS Statistics 25.

Clinical validity

We assessed whether CarerQol-7D and CarerQol-VAS scores were responsive to different caregiving situations. For categorical variables, we analysed whether CarerQol-7D and CarerQol-VAS scores differed significantly by relevant characteristics of the caregiving situation: the nature of health problems of the care recipient (mainly mental, mainly physical, or both); the health status of the care recipient; stability of the health status of the care recipient; relationship to the care recipient; duration of caregiving (less than 6 months; 7–12 months; more than 1 year); and, finally, if the care recipient lived in the same household as the respondent.

We calculated correlations between CarerQol-7D scores, CarerQol-VAS scores and the health status of the care recipient (EQ-5D-5L index score and EQ VAS Proxy versions), and the weekly time allocated to the caregiving tasks. We also explored whether individual EQ-5D-5L domain scores were associated with the CarerQol-7D and CarerQol-VAS scores.

Based on the results of previous validation studies, we expected significant association of CarerQol-7D and CarerQol-VAS scores with the care recipients’ health status, with lower scores associated with worse health status [16, 19] and with mental health problems [7]. Moreover, we expected the type of relationship between the caregiver and care recipient to be associated with CarerQoL-7D and CarerQol-VAS scores [7, 12, 19], lower scores with longer duration of caregiving [12, 16], and lower scores in case the caregiver and care recipient live in the same household [7], ceteris paribus.

Discriminant validity

We examined whether scores for the CarerQol-VAS, EQ-5D-5L index and EQ VAS of care recipients on the three different levels of the seven CarerQol-7D domains (no/some/a lot of problems) differed significantly. Based on previous studies, we expect significantly higher CarerQol-VAS score when the caregiver reports no problems with the caregiving domains, or more fulfilment and support [19].

Convergent validity

First, we expected positive associations of CarerQol-VAS scores with the two positive CarerQol-7D domains (fulfilment and support) and negative associations with the five negative CarerQol-7D domains [19]. Convergent validity of the CarerQol-7D was also evaluated by studying the associations between CarerQol-7D scores and CarerQol-VAS scores and EQ-5D-5L and EQ VAS scores of caregivers. Second, we examined the correlations between the CarerQol-7D and EQ-5D-5L domains. We expected a high correlation between the “Mental health problems” domain of the CarerQol-7D questionnaire and the “Anxiety/depression” domain of the EQ-5D-5L questionnaire. Also, we expected a significant association between the “Physical health problems” domain of the CarerQol-7D questionnaire and the “Mobility” and “pain/discomfort” domains of the EQ-5D-5L questionnaire. Thirdly, we compared CarerQol-7D and CarerQol-VAS scores to the overall caregiving experience of caregivers. We expected higher scores to be associated with more positive care experiences.

Results

Among the 1,000 respondents (female 51.2%; mean age 53.2, SD 15.2), 149 informal caregivers were identified [29]. The average age of caregivers was significantly higher than that of the general population in our sample (56.1 years, SD 14.2 vs 52.6, SD 15.2, respectively), while the share of women (58.4% vs 49.9%, p = 0.057) was not significantly higher. The average household size of caregivers was 2.4 persons (SD 1.3), and their per capita household income was 403 EUR (SD 245) and it did not differ significantly (p = 0.305) from that of the non-caregivers’ subgroup (conversion €1 = 322 HUF).

The average Carerqol-7D and CarerQol-VAS scores of caregivers were 76.0 (SD 16.2) and 6.8 (SD 2.3), respectively. CarerQol-7D and CarerQol-VAS scores did not differ significantly between sociodemographic subgroups but did differ with self-perceived health status (Table 1). The distribution of informal caregivers’ responses across the seven domains of the CarerQol-7D is presented in Fig. 1. The EQ-5D-5L and the EQ VAS scores of caregivers were 0.830 (SD 0.202) and 75.0 (SD 18.8), respectively, while among the care recipients these scores were 0.466 (SD 0.286) and 47.0 (SD 22.7), respectively. The distribution of health problems across the EQ-5D-5L domains for caregivers and care recipients are presented in a supplementary file (Online Resource 2).

Table 1 Sample characteristics, and CarerQol-7D and CarerQol-VAS scores by sociodemographic characteristics of caregivers
Full size table
Fig. 1
figure 1

Distribution of CarerQol-7D domain scores reported by informal caregivers. F female, M male

Full size image

Clinical validity

Table 2 presents the CarerQol-7D and CarerQol-VAS scores by characteristics of the caregiving situation. CarerQol-7D scores were significantly associated (p < 0.05) with the nature of the care recipients’ health problems, the lowest scores were found for caregivers of persons with mental health problems only (69.4). The difference between subgroups ‘mainly mental health problems’ and ‘both physical and mental health problems’ was not statistically significant. CarerQol-VAS scores also did not differ significantly across the three categories.

Table 2 CarerQol-7D and CarerQol-VAS scores by characteristics of caregiving situations
Full size table

CarerQol-7D scores were significantly associated with the health status proxy for the care recipient. When comparing subgroups, only CarerQol-7D scores for caregivers of persons with ‘Poor’ health status differed significantly from the other groups. CarerQol-VAS score was not significantly associated with the health status proxy scores of the care recipient.

We found the highest average CarerQol-7D score for caregivers of friends or neighbours (83.2, SD 18.2), and the lowest average score for caregivers of their partners (71.3, SD 14.6) (Table 2). CarerQol-7D scores were significantly higher in case the care recipient’s health status was expected to improve in the future, significantly lower (indicating higher burden) if the care was provided for less than 6 months (compared to > 6 months) and when the caregiver and care recipient lived in the same household (compared to those who do not live in the same household). Caregivers’ happiness (CarerQol-VAS scores) did not show significant differences between these categories.

Regarding correlations, we found small but significant correlations between CarerQol-7D scores, CarerQol-VAS scores and caregiving time (r = − 0.257 and − 0.212). The correlation between the CarerQol-7D scores and EQ-5D-5L scores (proxy) of the care recipients was small (r = 0.247) (Table 3). No significant correlations were found between the CarerQol-7D scores and EQ VAS scores (proxy) of the care recipients (r = 0.144), or between caregivers’ happiness (CarerQol-VAS scores) and care recipients’ health status (EQ-5D-5L score proxy: r = 0.054; EQ VAS proxy: r = 0.076). We found small but significant correlations between the CarerQol-7D scores and three (‘Usual activities’, ‘Pain/discomfort’, ‘Anxiety/depression’) out of the five domains of the care recipients’ EQ-5D-5L (proxy) questionnaire. We found no significant correlations between caregivers’ happiness (CarerQol-VAS) and care recipients’ EQ-5D-5L (proxy) domains. There were some small but significant correlations between some of the care recipients’ EQ-5D-5L domains and CarerQol-7D domains (Table 3).

Table 3 Spearman correlations of CarerQol-7D and CarerQol-VAS scores with EQ-5D-5L domains, EQ-5D-5L scores and EQ VAS scores of caregivers and care recipients
Full size table

Discriminant validity

Table 4 presents the CarerQol scores of respondents for the different levels per CarerQol-7D domains (e.g. no/some/a lot of problems). CarerQol-VAS scores were higher among caregivers that experienced fulfilment and received support and when problems were absent. However, differences between CarerQol-VAS scores were not significant for the domains ‘Problems with daily activities’, ‘Financial problems’ and ‘Support’. Both EQ-5D-5L scores and EQ VAS were significantly higher for caregivers with no mental health problems and no physical health problems.

Table 4 CarerQol and EQ-5D-5L scores of respondents on each level of the CarerQol-7D domains
Full size table

Convergent validity

CarerQol-VAS scores were positively associated with the positive domains of the CarerQol-7D, and negatively with the negative domains of CarerQol-7D. Correlation was significant between CarerQol-7D and CarerQol-VAS scores (r = 0.363). As shown in Table 3, the correlation between CarerQol-7D scores with EQ-5D-5L scores and EQ VAS scores of caregivers were significant and moderate (r = 0.453 and 0.387, respectively). The strength of correlation between CarerQol-VAS and EQ-5D-5L scores was significant and moderate (r = 0.453) and between CarerQol-VAS and EQ VAS scores was significant but small (r = 0.242). Both CarerQol-7D scores and CarerQol-VAS scores significantly correlated with all EQ-5D-5L domain scores (small to moderate correlations).

Regarding the correlations between domains of the CarerQol-7D and EQ-5D-5L instruments assessing the caregivers’ status, the highest correlations were found between the ‘mental health problems’ and the ‘anxiety and depression’, and the ‘physical health problems’ and ‘pain/discomfort’ (and ‘mobility’) domains, respectively (Table 3).

As Table 2 shows, caregivers’ CarerQol-7D and CarerQol-VAS scores differed significantly by the overall care experience. However, both for CarerQol-7D and CarerQol-VAS scores, this difference was not significant between the ‘neither negative nor positive’ and ‘rather positive’ categories. Similar relationships were found between overall care experience and the EQ-5D-5L scores of the care recipients.

Discussion

We developed the Hungarian language version of the CarerQol instrument and assessed its psychometric performance in a sample of informal caregivers in the general population. We evaluated the Hungarian version of the CarerQol instrument in relation to the health status of caregivers and care recipients, measured by the EQ-5D-5L questionnaire, and several characteristics of the caregiving situation. The analysis involved clinical, convergent and discriminant validity tests. This is the first validation study of this instrument in the Central and Eastern European region.

Regarding clinical validity, our findings are in line with results of previous validation studies [7, 12, 19]. First, we observed a negative association of the CarerQol-7D scores with the care recipients’ health status (measured by the EQ-5D-5L proxy score). Secondly, a negative association was observed with the duration of caregiving. Thirdly, similarly to findings by Brouwer et al. [7], lower CarerQol-7D scores were observed when the caregiver and the care recipient lived in the same household (compared to when this was not the case), and CarerQol-7D scores were also lower when the care receiver mainly suffered from mental health problems. Fourthly, similarly to Brouwer et al. [7] and Hoefman et al. [19], we also found associations between the CarerQol-7D and the type of relationship between the caregiver and care recipient. In our case, caring for a neighbour or friend was associated with higher care-related quality of life, while caring for a partner was associated with relatively lower scores. We can hypothesise that care recipients who are neighbours or friends may have additional helpers (family members, paid carers) as well. Probably, a spouse or partner may experience shared financial concerns or problems with the care recipient and may experience more care-related burden. Care time may also have effect on this relationship. These issues deserve further joint investigation in future studies involving larger samples. Some of the previous studies also reported relationship between CarerQol-VAS score and income [7, 19] or employment status [19]. However, we did not find a significant association between these variables in our sample. Nevertheless, the similarities to previous results strongly support the clinical validity of the Hungarian version of the instrument.

One strength of our study is that we included the overall CarerQol-7D index scores which have been applied before only in two validity tests [16, 20]. The CarerQol-7D index score performed better in distinguishing the burden of caregiving, than the general happiness measure, the CarerQol-VAS. Moreover, we focused on the convergent validity of the CarerQol-7D in relation to the EQ-5D-5L instrument, which relation has not been extensively studied before. With the EQ-5D-5L, we measured health-related quality of life of the caregivers and assessed if this was associated with their care-related quality of life. The significant moderate correlation between CarerQol-7D and EQ-5D-5L index score highlights a certain overlap between the two measures. Hence, when evaluating interventions for caregivers, there is a risk of double counting effects when both measures are used. On the other hand, the moderate correlation between CarerQol-7D and EQ-5D-5L proxy scores for care recipients indicates that estimation of CarerQol scores from EQ-5D-5L scores of care recipients should be done only with caution [36]. Alternatively, caregiving time could be estimated, and informal care included on the cost-side of an economic evaluation. Our EQ-5D-5L results moreover suggest that the prevalence of health problems is rather high among caregivers, especially in the ‘Pain/discomfort’ domain. This implies that a substantial proportion of caregivers had to cope with their own health problems while caring for someone else in their social network. Furthermore, we observed that in certain situations it was not the health status of the care recipient but other care-related factors (such as living arrangements, the relation to the care recipient, duration of care and expectations regarding future health improvements) that were associated with caregivers’ quality of life. These findings indicate that the context of caregiving should be considered as seriously as the health status of the care recipient when developing health and social care interventions that aim to support informal caregivers.

Limitations

Some limitations of our study need to be noted. The overall sample was representative in terms of main sociodemographic characteristics, but only up to the age of 65, thus only few elderly caregivers were included in our study. Hence, spousal caregiving, which is frequent in later life, as well as aspects of informal caregiving that are more typical in the elderly (e.g. the impact of their own disabilities and limitations, fear from the future) were probably only partly explored. Given that the survey was administered among an online panel, representativeness for the entire population is limited (selection bias). Informal caregivers who do not use internet could not be reached (coverage bias) and relatively few heavily burdened caregivers were involved (non-response bias). In our study, we did not consider whether the included caregivers were providing care for one or more individuals, or whether they were the primary caregivers of the care recipient or not. Even though our total sample of 1000 was sizeable, the sample size of the caregivers’ subgroup was relatively small (N = 149), which limits the statistical power of our analyses. Furthermore, health status of the care recipients was not assessed directly but by proxy measures. Finally, we could not apply Hungarian tariffs for the CarerQol and the EQ-5D-5L instruments, as these were not yet available.

Conclusions

The Hungarian language version of the CarerQol appears to be a valid instrument to assess the care-related quality of life of informal caregivers in Hungary, when applied in an online sample of informal caregivers in the general population. Strong similarities with validation studies from The Netherlands support the cross-cultural validity of the CarerQol. Further prospective studies are encouraged to replicate this finding in other groups of caregivers, including different (disease-specific) subgroups of caregivers and different caregiving situations. Moreover, future studies could assess the test–retest reliability and responsiveness of the instrument to changes. Our findings suggest that caregivers’ care-related quality of life and happiness are not perfectly correlated with the health status of the care recipient. Caregivers’ own general health and the context of caregiving, including personal factors and external conditions, seem to have more effect on care-related quality of life. This is relevant to consider when developing health and social care policy strategies that aim to support informal caregivers.