Background

The number of older people and their life expectancy has been rising steadily ranging from 50 years in resource poor to 83 years in resource rich regions [1]. Older people commonly have more than one chronic condition and have frequent encounters with healthcare providers [2]. Provision of care can be fragmented due to multiple assessments and treatments [3]. While focusing on a single condition may have advantages, a holistic approach with a review of outcomes that matter has greater value. Variation in outcomes of healthcare is a global challenge [4] and having the proposed set of outcome measures will facilitate and support reducing this variation.

Understanding what outcomes matter to patients would be valuable to clinicians and policymakers in aligning health care services to their needs. The aim of this project was to define a minimum set of outcomes for evaluating healthcare for older people. A Delphi technique was used to develop a balanced score card that was feasible to implement in routine clinical practice. An additional goal was to facilitate the creation of databases that can be compared and/or merged for analysis. This would support decision making being shared between providers, facilitate quality improvement and allow for benchmarking across organisations and countries.

The lack of outcome measurements that matter most to patients represents a barrier to health care improvement [5] and means providers have little information on which to judge the effectiveness of interventions. The ICHOM has to date developed 13 standard sets of outcome measures [6] and by 2017 at least 50% of the global disease burden will be covered. ICHOM (www.ICHOM.org) was founded in 2012 to promote value-based health care by defining global standard sets of outcome measures that matter to patients and promote adoption of these measures worldwide. This would be ICHOM’s first standard set of outcomes for a population as opposed to a specific condition such as cataracts, dementia or lung cancer [6].

ICHOM is a non-profit organisation supported by the Harvard Business School, Boston Consulting Group and the Karolinska Institute to transform health care systems worldwide by measuring and reporting patient outcomes in a standardised way. ICHOM organises global teams of physician leaders, outcomes researchers and patient advocates to define Standard Sets of outcomes per medical condition, and then drives adoption to enable health care providers globally to compare, learn, and improveA working group (WG) was organised by ICHOM, to represent a wide clinical, scientific and cultural background. Members (n = 31) included patient representatives, measurement experts, clinical, social and psychological researchers. Countries represented included Australia, Botswana, Canada, Germany, The Netherlands, Sweden, Switzerland, Taiwan, Peru, the United Kingdom, and the United States of America.

Method

A modified Delphi technique was used to develop the standard set. The Delphi technique is an iterative, multistage process to actively transform opinion into group consensus [7]. Over a period of 10 months, the working group met eight times over teleconferences.

The goals and scope of the working group were discussed in the first teleconference. The second to fourth teleconferences (call 1 to 3 in Fig. 1) focused on the outcome domains and definitions to include in the standard set. In preparation for teleconferences 2–4, the working group were provided with information from literature reviews (Additional file 1: Table S1) and an older person’s and carer focus groups (Table 1). ICHOM organised an older people focus group with six attendees (age range 68–89) after the working group launch, to obtain their perspectives, using open-ended questions. Participants, consulted through Age UK’s networks, discussed which outcomes were of greatest importance to them. Age UK (http://www.ageuk.org.uk) is a charity dedicated to improving the lives of older people via a national network supported and facilitated by partnerships.

Fig. 1
figure 1

Showing the process and timeline for the working group. A flow chart showing the working group process

Full size image
Table 1 Themes from the older persons and carer focus group
Full size table

To support the decision making process the working group used a set of 4 criteria; represent the end results or ‘outcomes’ of care, represent what is important to OP and their families, feasible to capture and can be used for quality improvement programmes.

The discussion content was collated into online surveys. Working group members were asked to submit their feedback and votes via a web survey questionnaire. The survey had all the outcomes discussed with the level of agreement ranked during the teleconferences. Decisions resulting from the surveys required a minimum 50% of the working group membership participation. It was anticipated that due to time zone differences and schedules, this was a practical and reasonable standard to adopt given a fixed deadline by which the work had to be completed.

Teleconferences 5 and 6 (calls 4 and 5 Fig. 1) addressed case mix factors and definitions. Teleconferences 7 and 8 (calls 6 and 7 Fig. 1) focused on reviewing the agreed outcome domains, case mix factors and how the standard set would be shared with the healthcare community. Over the 10 months of the project, attendance for the teleconference meetings ranged between 51.7% to 75.9% (mean 61.1%). Three voting surveys were conducted with varying response rates. For a measure to be accepted as an outcome the working group set a standard of 70% and above of members voting to include a measure as an outcome.The final standard set was approved by all members of the working group.

PRISMA reporting principles were used as guidance for the literature search strategy [8]. Titles, keywords and abstracts were searched using MeSH or equivalent terms in the following databases PubMed/Medline, EMBASE, Psychinfo, Social Care online, Cumulative Index to Nursing and Allied Health Literature (CINAHL), COCHRANE, PsychInfo. Inclusion criteria included: (Aged, 80 and over OR Frail elderly or Comorbidity) AND (quality of life OR outcome assessment (healthcare) OR quality indicators), Paper and guidelines reporting on patient-reported and patient-centred outcomes, English language abstracts, reviews and randomised controlled trial,2005 onwards. Exclusion criteria included Non-English language, irretrievable, insufficient outcome data, unclear diagnoses, unvalidated outcomes.

Additional sources of information included existing measurement approaches adopted by longitudinal ageing surveys [9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38]. Figure 1 summarises the working group process.

Triangulating findings from the literature review and focus group with the working group discussions would strengthen the resultant outcome measures decided upon and highlight the key issues that most matter to older people. Experience of and satisfaction with care by older people and their carers including distress and mood was noted in quality of life literature reviews but did not come up specifically in the focus group discussions.

A three tiered hierarchy framework [39] has been utilised to categorise the outcome measures. Tier 1 is the health status achieved or retained with survival and then degree of recovery achieved. Tier 2 is the process of recovery with time to recovery and return to normal activities as well as the treatment burden such as side effects and complications.Tier 3 is sustainability of health with recurrences and long term consequences of care interventions.

A specific cut off age was considered inappropriate due to the range in life expectancies around the world. During the working group discussions, it was agreed that the last 10 years of life captured a period in which a person might be regarded as being old across the world and potentially seeking healthcare. Therefore, rather than specifying a fixed cut-off age as the inclusion population for this standard set, the working group recommended subtracting 10 years from the estimated life expectancy at 60 years in each country or region. The inclusion population would be those who are at or above this age. For example, in South Africa, the life expectancy at age 60 is 76 years old, therefore the inclusion population would be all those over the age of 66 [40,41,42,43]. These can be utilised for any society in the world where a particular age is viewed as old if it does not fall within the definition above. The principles that apply to older people would be the same. This respects and accepts that each society can define what old age is to them.

Results

The suggested initial outcomes were chosen based on congruency across findings from the registries, surveys, literature searches and engagement with older people. A minority were chosen based on the consensus experience of the working group members. In the general category health status, quality of life, mortality, independence, remaining at home, carer health, and autonomy were deemed essential. In physical health, functional status, symptom occurrence, sleep, harm, frailty stage, nutrition, weight loss was also essential. Mental and psychological health had cognition, mood and loneliness as essential. Social network, support and isolation were essential in the social and community category. Length of stay, care coordination and discharge to place of choice were essential in healthcare utilisation. Dignity, shared decision making, access to information and advice were deemed essential under the experience/process category.

Tier 1 outcomes were overall survival, frailty and place of death. Tier 2 outcomes were polypharmacy, falls, participation in decision making and time spent in hospital. Tier 3 outcomes included loneliness and isolation, activities of daily living, pain, mood and emotional health, autonomy and control and carer burden. The results of the voting outcomes are summarised in Tables 2, 3, 4 and 5 summarises the outcome measures mapped to the tiers.

Table 2 Showing voting outcomes following round 1 survey of working group members. The % refers to the proportion of those who voted in support of each item
Full size table
Table 3 Showing voting outcomes following round 2 survey of working group members. The % refers to the proportion of those who voted in support of each item
Full size table
Table 4 Showing voting outcomes following round 3 survey of working group members. The % refers to the proportion of those who voted in support of each item
Full size table
Table 5 Standard Set of Outcome Domains for Older People
Full size table

The collection of a minimum set of baseline characteristics is recommended to allow case-mix adjustments [44, 45] Case-mix adjustment is a useful and fair way for making comparisons among health care providers. Taking these into consideration reduces disadvantages in comparative ratings due to differences in the underlying population of interest.

The working group agreed:

  1. a)

    Demographic factors: Such as age, gender, level of education, living arrangements, marital status and ethnicity. Items are harmonised to other ICHOM surveys. The educational level should be assessed following the International Standard Classification of Education [46] to allow global comparisons.

  2. b)

    Condition specific variables: These were frailty stage, type of medication used, total number of medications and baseline cognition.

  3. c)

    Systemic variables: Included were co-morbidities, smoking, alcohol use, weight, height, body mass index, vision and hearing impairment, and baseline activities of daily living.

A reference guide is freely available online that further describes the recommended instruments, data sources and provides detailed information (www.ichom.org).

Discussion

A standard set of outcome measures that matter to older people has been developed by a global panel of interdisciplinary professionals,older people and their carers.

The strengths of this project include the global interdisciplinary collaboration, involving older people and their carers and triangulating findings from a focus group, professional experience and the published literature. Obtaining information from various sources was important as not surprisingly not all domains were articulated in the single focus group due to its small sample. This also focused on a subset of a population rather than on a specific medical condition. To date no other set of outcome measures for older people has been developed using this approach. This approach has reduced the chances of excluding important themes that matter to older people. In attempting to be comprehensive and for the findings to be feasible for implementation, some themes had to be excluded. This does not mean they are not important but feasibility of the outcomes being used was regarded by the working group to be critical. The outcome measures have not been developed for use by academic researchers and will therefore not meet criteria for use by that group. The measures have been specifically developed for practical use by healthcare providers and those who pay for these services.

The framework utilised to develop these outcomes is based on Porter’s outcome hierarchy [39]. Tier 1 is the most important with the outcome being survival or the best possible state achieved for a condition. Tier 2 outcomes are the issues related to achieving tier 1 outcomes such as the time to recovery from a flare up of a chronic disease or recovery from an acute disease. Included in this tier 2 are all the harms associated with investigations and treatment. Tier 3 outcomes relate to long term health status.

Healthcare providers should appreciate and understand the perception, attitude and behaviour of those they care for [47]. In this context, “what matters to you” as a recipient of healthcare is more important than “what is the matter with you.” We have attempted to balance the information derived from previous studies to compensate for this by incorporating the views of OP and their carers. We hope that whilst not ideal, concerted efforts were made to ensure that the voice of OP and their carers were incorporated.

The value of performance based measures including grip strength as health outcomes for older adults [48] was discussed. The evidence base supporting the value of such measures for providing integrative assessments of older persons’ health, and for identifying persons at risk of a decline in health was recognized. The majority of the group considered the collection of such measures burdensome as part of a minimum set of indicators to be included in the standard set but endorse the value of incorporating them in specialty geriatric settings.

Frailty is well recognised [49, 50]. For providers, understanding the proportion of those becoming frail will aid their future resource allocation, service planning and prevention strategies [51, 52]. There was agreement for a frailty measure as a risk factor for outcome measure adjustment but much less agreement concerning the role of a frailty measure as a service outcome. Indeed, this was the most discussed topic.While the phenotype model [53] remains the gold standard for diagnosing frailty, the cumulative deficit model [54] was viewed by a majority as what clinicians will identify with more easily. Both have been validated in aiding clinical decision making [48, 55] and [56]. The Canadian Study of Health and Ageing (CSHA) Clinical Frailty Scale [43] was recommended as the tool to be used in the standard set to assess frailty. It mirrors clinical judgement, is objective [57] and can be used in places with no electronic health records. However, alternative frailty tools may become widely implemented in some countries. For example, an electronic frailty index is now available for use for over 90% of general practitioners in England [58] (http://ageing.oxfordjournals.org/content/early/2016/03/03/ageing.afw039.full) and, an online tool (www.johnshopkinssolutions.com/solution/frailty) is available for frailty assessment utilising the phenotype model.

At first glance, polypharmacy, falls and length of stay in hospital may not appear to be outcome measures. This is where triangulation of findings from focus group and the working group discussions added value to this project. These three areas were things that mattered to older people, their carers and clinicians. It was felt that without keeping track of these in the form of outcome measures it could easily fall off the radar of health systems caring for older people. The SF-36 and other tools to capture the metrics around the outcome measures were chosen solely for very practical reasons. It had to be free to use and cover as many of the outcome measures to reduce the number of tools and complexity of use associated with this.

The final set of outcome measures arrived at has been reduced down from the original set at the outset of the project. In settling for a cut off, the working group applied feasibility and comprehensiveness as a guiding principle. In using such a diverse group, it is hoped that a reasonable balance has been struck.

The working group consensus was to measure the standard set outcomes longitudinally over time. A minimum annual frequency was recommended given the challenges of measurement and capturing population level changes. It was acknowledged that while some stakeholders might be interested and keen to collect these data more frequently and / or at each healthcare encounter, to recommend more than an annual collection could be too prescriptive and burdensome for providers.

This was an ambitious project and the working group recognised that it was unlikely to satisfy everyone. This is however a good starting point and further outcome measures should be explored and developed for specific niche groups such as older people with frailty, cognitive impairment, physical disability as well as exploring outcome measures that would be relevant for carers and researchers in old age health. Furthermore as these outcome measures start being used, areas for improveing them would arise and allow for them to be amended continuously to make them relevant and fit for purpose as our healthcare environment continues to change.

Conclusion

Through the efforts reported in this paper, the ICHOM older people working group defined a standard set of recommended outcome measures that matter to older people. This is a first effort towards a standardisation of outcome measures to improve the quality of care for older people. Much further work remains to be done but in the meantime, itwould be ideal for national data sets to include information which allows these outcomes to be derived routinely.