A systematic review of outcome measures for economic evaluation in aged care

Background

In the year 2000, 11 % (605 million) of the world’s population was over 60 years of age and this figure is forecast to rise to 21 % (2 billion persons) by 2050 [1]. The fastest growing cohort in this population is the oldest old (over 80 years of age) accounting for 14 % of the older people population in 2014 and expected to increases to over 19 % by 2050 [1]. Older people currently represent the fastest growing age group in most developed countries and are major users of health and aged care services. Aged care services have an important role to play in enhancing the self-worth, independence and quality of life of older people [2, 3]. The projected exponential increase in the prevalence of older people living in the community with cognitive decline, frailty and other co-morbidities will inevitably contribute to a significant increase in the demand for, and utilization of aged care services in the future [1]. Most governments in developed countries subsidise different types of aged care services to enable older people to remain living at home including home care packages to support activities of daily living, nursing care, meals services, adult day care services, equipment and home adaptations, re-ablement services (to assist older people in recovering from and adapting to physical and mental illness) and support for people living with dementia [4–7].

In measuring the impact of service innovations in aged care, researchers in health economics and other disciplines are increasingly recognising that quality of life is a multi-dimensional concept and the impact of interventions for older people goes beyond health status, incorporating psychosocial and emotional well-being, independence, personal beliefs, material well-being and the external environment that influences development and activity [8–10]. Older people’s interpretation of quality of life is based on their capability to achieve those things or participate in activities they value, viewing health as a resource to facilitate their participation in activities of daily living and social interactions [10–14]. The value they obtain from health care services and other interventions goes beyond physical functioning or the health dimensions, as measured by health related quality of life (HRQOL) instruments, to include non-health dimensions such as security in their physical environment, independence, sense of value and attachment, which are only captured by broader instruments [15, 16]. It is therefore important that instruments used to measure and value quality of life outcomes in the aged care sector capture such broader quality of life outcomes.

Instruments for measuring health status and/or quality of life may be differentiated into preference based and non-preference based. Preference based instruments typically incorporate scoring algorithms which are based upon the preferences of a general population sample for the health and/or quality of life states defined by the instrument elicited using one or more valuation methods such as the visual analogue scale (VAS), time trade off (TTO), standard gamble (SG) and discrete choice experiments (DCE) [17, 18]. Preference based instruments are typically used by health economists and health service researchers within economic evaluations in a cost utility analysis framework (CUA) where the main measure of outcome is quality adjusted life years (QALYs). Non-preference based instruments are not suitable for application in CUA because they do not facilitate the calculation of QALYs. Table 1 summarises some of the most popular generic preference based and generic non-preference based instruments.

Table 1 Generic preference based and non-preference based instruments
Full size table

In contrast to generic preference based instruments, condition specific and population specific preference based instruments focus upon one condition or disease area or population of interest. Population specific preference based instruments have been designed to be utilised with a single population group e.g. children or older people. Examples of population specific preference based instruments include the Adult Social Care Outcomes Toolkit (ASCOT) [19, 20] designed to measure quality of life for individuals receiving social care and the older person specific ICEpop CAPability measure for Older people (ICECAP-O), a measure of capability [12]. Table 2 summarises some of the most popular older person specific instruments.

Table 2 Older people specific instruments
Full size table

This paper describes the methods and results of a systematic review to identify instruments that have been used in measuring quality of life outcomes in older people and documents the contexts in which the instruments have been applied. The primary focus is on instruments suitable for application within a CUA in the aged care sector. The findings from this review will be utilised to inform the design of economic evaluations in community aged care service delivery in Australia. The findings also have wider applicability internationally for researchers designing and conducting economic evaluations with dependent older people in determining the most suitable instruments for application in the aged care sector.

The focus on older-people-specific outcomes is motivated by assertions in the literature that economic evaluations of interventions aimed at older people should be conducted using outcome measures tailored towards meeting the goals of services consumed by older people [16]. Research also indicates that such measures need to capture broader quality of life outcomes such as affection and control that extend beyond health related quality of life whilst also recognising that many older people view health as a resource to facilitate their participation in activities of daily living and social interactions [10, 11, 15]. Different age groups of the population have also been reported to prioritize different areas of life as important, with older people being the most likely to prioritize their health and ability to get out while younger people are more likely to prioritize work, finances, having chances to learn new skills and their sex lives [21–23].

A recent systematic review conducted by Makai and colleagues [15] identified instruments suitable for applications in economic evaluations of interventions in older people receiving long term care. This systematic review builds upon the work previously conducted in two main ways; firstly by capturing the three year period beyond their systematic review and secondly by focusing in more detail upon the particular contexts in which the instrument/s were applied.

Capturing information on and gaining insights into the context within which instruments have been previously applied is particularly relevant in guiding the selection of the most appropriate instrument/s for application within economic evaluation in the aged care sector [24]. Some quality of life instruments have been specifically developed for use within certain settings. For instance, the developers of the ASCOT originally designed the instrument to capture information about an individual’s social-care-related quality of life within community and residential settings [19]. It could also be argued that health-related quality of life instruments such as the EuroQoL 5 dimensions (EQ-5D) are more suitable for individuals receiving health-focussed interventions such as those in hospital where the primary objective is the maintenance of or improvement in health. There is evidence to indicate that there are differences in quality of life perceptions between hospitalised/ambulatory and non-hospitalised older adults [23]. Fassino et al. [25] also showed that aspects of quality of life that matter to dependent older people (individuals dependent on others for their day-to-day living) differ from those that matter to independent older people and a study by Bowling et al. [26] found that better functional ability was related to better quality of life in older age. Bowling et al. [27] also postulated that the multifaceted nature of independence, particularly in older age, is mostly ignored in the wider quality of life measurement literature.

In this study, therefore, we sought to assess whether quality of life instrument use differed according to context, which was defined by the location or setting in which services for older people were being provided, i.e. in the community, residential facilities or within a hospital and according to the level of dependency for older people living in the community (specifically whether the study population was made up of dependent or independent older people). For the purposes of this review, dependency was defined as frailty or individuals dependent in activities of daily living as assessed by instruments such as the Barthel index and individuals who required or lived with an informal carer such as older people with cognitive impairment and those who have experienced or are recovering from stroke. Studies where majority of the study population was comprised of dependent older people were classified under the dependent heading.

Therefore, this paper seeks to provide arguments for the suitability (or otherwise) of the different instruments in economic evaluations of interventions for older people in various contexts within the aged care sector.

Specifically, the three main objectives of the review were:

  • To identify instruments used in the published literature to measure quality of life outcomes for older people

  • To identify the different contexts in which the instruments have been used

  • To provide arguments for the appropriateness and suitability of the different quality of life instruments within a cost utility analysis (CUA) framework of service delivery innovations in aged care.

Review

Methods

The key search questions to be answered by this review were consistent with the three main objectives previously specified. The review process was consistent with the PRISMA guidelines for the conduct of systematic reviews [28].

Databases

PubMed, Medline, CINAHL, Scopus, and Embase, PsycInfo, informit and Web of science.

Search terms

Keywords were replicated based on the review undertaken by Makai et al. [15] with the addition of two more concepts (instruments and the contexts in which quality of life was measured) as well as appropriate subject headings and keywords based on the objectives of this review. Five major concepts were applied in this search; quality of life, the population (older people aged 65 years and over), validity, instruments and study contexts defined as community aged care or residential aged care. These concepts were combined with the ‘AND’ operator. The full search strategy, including subject headings and key words, used in Medline is attached in Additional file 1. The same broad strategy was replicated in other databases with appropriate adjustments made to align the strategy to the requirements of these other databases.

Selection criteria

Studies that met the following criteria were considered: 1) measured quality of life and/or health status and/or health related quality of life as a primary or secondary outcome, either as a snapshot/cross-sectional or longitudinally over time in aged care settings, 2) used a generic or older person specific preference based instrument or a non-preference based older person specific quality of life instrument or both 3) study population was exclusively 65 years and over, dependent older people living in the community or in residential aged care facilities, and 4) published in peer reviewed journals in the English language between 2000 and July 2015.

Studies were excluded if 1) study population was not exclusive to people aged 65 years and over 2) study population was focused primarily upon patients in the health system and/or not comprised of dependent older people living in the community or in residential aged care facilities 3) only disease specific or generic non-preference based measures of quality of life/health related quality of life were used and studies in which quality of life was not measured using an instrument or they used questionnaires specifically designed for the study, 4) dissertations, commentaries, conference papers or review articles and studies for which the full text article could not be obtained.

To assess the reliability of the study selection process, selection was performed by all three authors on a random sample of 5 % of the studies by using the selection criteria described above. The overall agreement was then calculated using Cohen’s kappa statistic [26].

Results

Study selection process

Figure 1 presents the study selection process which was divided into four key stages:

Fig. 1
figure 1

Study selection process

Full size image
  1. i)

    Identification: In July 2015, 9545 studies were identified from the online databases and an additional 337 studies from backward/forward searches and basic internet search using the key words. 4969 studies were eliminated because they were found to be duplicates.

  2. ii)

    Screening: 4913 titles and abstracts were screened for eligibility. 4322 studies were excluded as they did not meet the eligibility criteria.

  3. iii)

    Eligibility: 591 full texts articles were assessed at this stage. All three authors independently assessed 5 % of the identified studies and overall agreement was then calculated using Cohen’s kappa statistic [28] 380 studies were excluded because they measured quality of life using generic non-preference based instruments while 63 studies used questionnaires specifically designed for the study, the study population in 10 studies was not exclusive to people over 65 years of age and the full text articles could not be obtained for five studies. A further 97 studies were eliminated whose study population was focused primarily upon patients in the health system and/or not comprised of dependent older people living in the community or in residential aged care facilities.

  4. iv)

    Included: 36 studies were considered in the qualitative synthesis; 26 studies undertaken among dependent older people receiving community aged care services and 10 studies among those residing in aged care facilities. The chance-corrected agreement between the abstracts selected by the primary author and the two co-authors was in the range of 0.77 and 0.88, with an average kappa statistic of 0.81, which was substantial/almost perfect [29].

Key finding 1: study characteristics

Full reports of the included studies were read to extract information relating to the population and country of the research, sample size and type of study, the instrument used and the context in which the instrument/s were applied. Details of all the studies assessed for eligibility and classified by context in this review are provided in Table 3 .

Table 3 Classification of studies by context
Full size table

Geographically studies were conducted in several countries; 7 (19 %) from the Netherlands, 6 studies (16 %) each from the UK and Canada, 5 studies (14 %) from the USA(with one study from both USA and Canada) while three studies each (8 %) were from Australia and Italy. 2 studies (5 %) were conducted in Sweden and one study (3 %) each in Poland, Germany, Denmark, Japan and Turkey. The majority of studies (59 %) were undertaken in Europe. Figure 2 is a summary of the geographical distribution of the included studies.

Fig. 2
figure 2

Geographical distribution of studies

Full size image

Twenty two studies (61 %) were undertaken using a cross-sectional design, 8 (22 %) randomised control trials, 3 (8 %) prospective cohort studies, 2 (6 %) longitudinal studies and one explorative qualitative study.

The sample sizes varied substantially from a minimum of 10 to a maximum of 29,935 older people. Figure 3 summarises the sample size distributions of included studies.

Fig. 3
figure 3

Sample size distribution

Full size image

Key finding 2: contexts and settings

The identified studies were grouped into four contexts; community based dependent, community based independent, residential facility based and hospital based older people (see Table 3). Two contexts were considered to be reflective of older people receiving aged care services; community based dependent older people and older people based in residential aged care facilities. The results reported below therefore relate to these two contexts:

  1. a.

    Community living dependent older people: Twenty six studies were identified; seven studies conducted in older people with no particular prevailing condition [30–36], ten studies among older people with cognitive impairment [37–46] while seven studies comprised of frail older people where cognitive status was unspecified [24, 47–52]. One study each was among older people with depression [53] and those with a previous stroke [54].

  2. b.

    Residential aged care context: Ten studies were identified in this context; three among residents with cognitive impairment [55–57], two studies among frail older residents where cognitive status was unspecified [58, 59] and four studies recruited samples from the general resident population who had no cognitive impairment and were not too ill to participate in the study [60–63] while one study specifically considered residents with depression [64].

Key finding 3: instruments used to measure quality of life

Table 4 summarises the instruments that have been used to date to measure quality of life within the community and residential aged care context and Table 5 reports the frequency with which they were used.

Table 4 Instruments used in the identified studies
Full size table
Table 5 Frequency with which the different instruments were used
Full size table

In general, the most commonly applied generic preference based instrument was the EQ-5D (51 %) followed by the ASCOT (16 %), and the most widely applied older person specific instrument was the ICECAP-O (11 %) (Table 5). In the community aged care context, the most applied generic instrument was the EQ-5D (n = 16) followed by the ASCOT (n = 6) and HUI3 (n = 5) and the older people specific ICECAP-O (n = 3). Other instruments applied in this sector were the older people specific OPQOL (n = 2) and CASP-19 (n = 1) and the generic QWB (n = 1). In the residential aged care context on the other hand, the most widely applied instrument was the EQ-5D (n = 7) followed by the ICECAP-O (n = 2). Other instruments applied were the ASCOT (n = 1) and the older people specific WHOQoL-Old (n = 1).

The popularity of the EQ-5D may be attributed to several reasons including its brevity, the availability of various translations and scoring algorithms from several cultures and countries worldwide and its recommended use for the economic evaluation of new technologies by the National Institute for Health and Care Excellence (NICE) in the UK [65]. The ICECAP-O, CASP-19, OPQOL and ASCOT are all relatively newer instruments developed and validated in the UK. NICE recommends the ASCOT as the preferred measure for outcomes in social care and the ICECAP-O where outcomes in terms of capabilities are to be measured [65].

Five studies explicitly measured quality of life at more than one time point allowing some assessment of the sensitivity to change over time to be made [24, 45, 51, 63, 64]. Absolute changes in utility scores ranged from 0.003 to 0.21 based on 6-18 month follow-up periods. There is no consensus in the literature about what the minimal important difference (MID) should be (i.e. values range from 0.03 [66–68] to 0.074 [69] for changes in EQ-5D [64] and HUI3 utilities [45]). Some of the changes in quality of life reported upon in these studies were larger than the MID value reported in the literature (0.03). However, Drummond [66] indicates that if outcomes based on preference-based measures are to be used to influence resource allocation decisions, it is the difference in cost-effectiveness, such as the incremental cost per QALY, rather than the change in quality of life that is important. This therefore means that within an economic evaluation framework it is important to consider changes in quality of life in addition to the cost of bringing about such changes [66].

Discussion

In considering the suitability of instruments for use in evaluating interventions in the aged care sector, it’s important to consider the aspects of quality of life that are most important to older people and to assess the ability (or otherwise) of each instrument to capture these aspects within the framework of economic evaluation.

This review has highlighted the multi-dimensional nature of quality of life for older people. Key elements of quality of life amongst community living dependent older people include physical and cognitive functioning [32, 34], independence in activities of daily living [37, 47], social relationships [31–33, 47], absence of morbidity or health impairments [32, 33, 51] and pyscho-social wellbeing [32, 38, 51] as well as social connectedness and accessibility within the home and community [36, 47, 50]. In the residential aged care context on the other hand key contributing factors to quality of life include independence in activities of daily living, sense of dignity and physical freedom [55, 58, 60, 70], absence of morbidity or health impairments [58] and happiness coupled with social participation [60, 61].

These findings are consistent with arguments made by several commentators that social participation [71–73], health [10, 71, 72], wealth [72–74], home and community environment [10, 72–75], and independence or control over their life [10, 72, 73] represent key dimensions for any assessment of quality of life in samples of community living dependent older people. In the residential aged care context, important dimensions highlighted in the literature include social participation in family and leisure activities [76–78], independence [76–79], peace and contentment [76, 79], security [77, 79] and spiritual well-being [77, 78].

Generic preference based instruments assess respondents’ level of physical functioning through domains such as mobility within the EQ-5D, 15D and HUI3 and independent living within the AQOL. Psychological and emotional wellbeing is accounted for by anxiety/depression on the EQ-5D and 15D, emotion on HUI3 and happiness and mental health of the AQOL. Relationships and family dimensions may be captured by the relationships domain of the AQOL-8D. However, the question remains as to whether these quality of life dimensions are interpreted in the same way by older people themselves. For example physical functioning in older people may not necessarily be linked solely to their levels of mobility but also to their ability to participate in meaningful activities that emphasise their dignity, independence and relevance to society or their significant relations [58, 71].

Of the preference based instruments identified by this review, the EQ-5D is relatively easy to administer and has a higher completion rate [49, 70, 80–82]. With consideration to respondent burden, the EQ-5D may be considered to have practical advantages as it is relatively brief with only 5 dimensions. The other generic preference based instruments have more dimensions and/or dimension levels as illustrated in Table 1 with several also having mixed response items which may be considered to impose an additional response burden. Research has however shown that the EQ-5D has higher ceiling effects when compared against other instruments such the SF-6D and this needs to be taken into account [49, 83, 84]. The recent development of the new five-level version of the EQ-5D may however minimize this ceiling effect [85, 86]. Coast et al. [87] and Hulme [88] have advocated for interviewer help to complete the instrument when used among the very elderly and those with reduced cognitive function. In fact other researchers have used proxy respondents to apply the instrument to people with mild to moderate cognitive impairment [38, 42, 63, 70]. As highlighted previously, the vast majority of preference based HRQOL instruments such as the EQ-5D were developed for application in a health care context and are narrowly focused on health status alone a dimension highlighted in both community and residential aged care contexts, but it may not be the most appropriate indicator for measuring quality of life in older people. Several researchers have argued that these instruments are unlikely to be appropriate for assessing the well-being of older people in an aged care context because broader quality of life dimensions are important in this context.

The ASCOT is a preference based measure designed to specifically measure social care related quality of life and captures dimensions of quality of life relevant to people receiving social care services [19]. The ASCOT necessarily takes a broader quality of life focus including dimensions such as dignity, safety, control over daily life and social participation which are important for older people in both community and residential aged care [24, 49]. The ASCOT may therefore be considered a relevant instrument to apply when assessing quality of life in relation to service innovations in the aged care sector.

The older person specific instruments identified by this review reflect quality of life in a broader sense and thereby tend to address the majority of key domains previously highlighted as important to older people.

The OPQOL may be considered to represent the most comprehensive older people specific instrument developed to date as it contains quality of life domains/dimensions identified as important for both community and residential aged care contexts and it incorporates both health status and broader quality of life domains [89–91]. However, the OPQOL currently has limited use in an economic evaluation framework because it is not preference based.

The ICECAP-O also encompasses quality of life dimensions that are relevant in both community and residential aged care contexts such as independence or control, security, social participation or attachment, and it has been validated for use in older people in the health and the aged care sectors in several European countries [16, 49, 55, 70, 92]. It is also notable that good construct validity has been reported for the ICECAP-O when used in older people with mild to moderate cognitive impairment, a significant cohort of older people in general [24, 55, 70, 93]. The ICECAP-O is also preference based which potentially facilitates its use in economic evaluations. Some commentators have suggested that the ICECAP-O is not suitable for use in CUA because of its focus on capabilities which does not enable the calculation of QALYs [24, 94]. However, other commentators have indicated that the ICECAP-O may be used within the framework of economic evaluation, with a revised capability based methodology for capturing the benefits associated with new interventions and/or service innovations [95, 96].

Overall for both the community and residential aged care contexts it’s important to emphasize the breadth of dimensions that affect older people’s quality of life. Compared to the EQ-5D, although the ASCOT and ICECAP-O do not have a health dimension per se, they are more sensitive to change and are more associated with broader quality of life beyond health [24]. This review argues that the choice of instrument is determined by the objective of the intervention being assessed; the EQ-5D being preferred for interventions aimed at maintaining health while the ASCOT and ICECAP-O are preferred for interventions with broader benefits beyond health such as service delivery innovations in the aged care sector [24, 49, 55].

A limitation of this study was that due to the heterogeneity of and the lack of adequate data from the studies included in our sample, it was not possible to conduct any meta-correlations or meta-regressions to empirically test whether the instruments used in the studies included in this review perform differently in various contexts.

Conclusions

This review has highlighted that for older people quality of life is a multi-dimensional concept, being defined by broader dimensions of quality of life in addition to health status. Older people typically derive wider quality of life benefits from service innovations in aged care that may or may not also have a positive impact upon health status. In order to reflect the multi-dimensionality of quality of life and to capture wider quality of life benefits within an economic evaluation framework the most appropriate quality of life instrument for application in the aged care sector is one that ideally measures not only health status and functional ability but also wider quality of life dimensions of importance to older people such as independence, psychological wellbeing, social relationships and social connectedness.

Currently no single instrument exists which is preference based and commensurate with the QALY scale (and therefore appropriate for application in economic evaluation) incorporating both health status and the broader elements of quality of life previously highlighted.

In the absence of a single ideal instrument for CUA to assess the cost effectiveness of service innovations in the aged care sector, this review recommends the use of a generic preference based instrument, the EQ-5D to obtain QALYs in combination with the ICECAP-O or the ASCOT to facilitate the measurement and valuation of broader quality of life benefits as defined by older people.